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equipment are necessarily the best available for the purpose. Nor does such identification imply a finding of fault or 
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Disclaimer No. 3 

Pursuant to section 7 of the National Construction Safety Team Act, the NIST Director has determined that certain 
evidence received by NIST in the course of this Investigation is "voluntarily provided safety-related information" that is 
"not directly related to the building failure being investigated" and that "disclosure of that information would inhibit the 
voluntary provision of that type of information" (15 USC 7306c). 

In addition, a substantial portion of the evidence collected by NIST in the course of the Investigation has been 
provided to NIST under nondisclosure agreements. 

Disclaimer No. 4 
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since, due to the destruction of the WTC buildings, NIST could not verify the actual (or as-built) construction, the 
properties and condition of the materials used, or changes to the original construction made over the life of the 
buildings. In addition, NIST could not verify the interpretations of codes used by applicable authorities in determining 
compliance when implementing building codes. Where an Investigation report states whether a system was 
designed or installed as required by a code provision, NIST has documentary or anecdotal evidence indicating 
whether the requirement was met, or NIST has independently conducted tests or analyses indicating whether the 
requirement was met. 

Use in Legal Proceedings 

No part of any report resulting from a NIST investigation into a structural failure or from an investigation under the 
National Construction Safety Team Act may be used in any suit or action for damages arising out of any matter 
mentioned in such report (15 USC 281a; as amended by P.L. 107-231). 
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Abstract 



This report documents the methods and outcomes of the telephone survey, face-to-face interviews, and 
focus groups that were conducted in support of the federal building and fire safety investigation of the 
World Trade Center disaster. In total, 803 telephone interviews were completed, 220 face-to-face 
interviews were completed, and 6 focus groups involving 28 individuals were completed. All interview 
material was transferred to National Institute of Standards and Technology investigators for analysis. 

Keywords: Survey, face-to-face interviews, focus groups, World Trade Center. 
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Genesis of This Investigation 

Immediately following the terrorist attack on the World Trade Center (WTC) on September 11, 2001, the 
Federal Emergency Management Agency (FEMA) and the American Society of Civil Engineers began 
planning a building performance study of the disaster. The week of October 7, as soon as the rescue and 
search efforts ceased, the Building Performance Study Team went to the site and began its assessment. 
This was to be a brief effort, as the study team consisted of experts who largely volunteered their time 
away from their other professional commitments. The Building Performance Study Team issued its 
report in May 2002, fulfilling its goal "to determine probable failure mechanisms and to identify areas of 
future investigation that could lead to practical measures for improving the damage resistance of buildings 
against such unforeseen events." 

On August 21, 2002, with funding from the U.S. Congress through FEMA, the National Institute of 
Standards and Technology (NIST) announced its building and fire safety investigation of the WTC 
disaster. On October 1, 2002, the National Construction Safety Team Act (Public Law 107-231), was 
signed into law. The NIST WTC Investigation was conducted under the authority of the National 
Construction Safety Team Act. 

The goals of the investigation of the WTC disaster were: 

• To investigate the building construction, the materials used, and the technical conditions that 
contributed to the outcome of the WTC disaster. 

• To serve as the basis for: 

- Improvements in the way buildings are designed, constructed, maintained, and used; 

- Improved tools and guidance for industry and safety officials; 

- Recommended revisions to current codes, standards, and practices; and 

- Improved public safety. 

The specific objectives were: 

1. Determine why and how WTC 1 and WTC 2 collapsed following the initial impacts of the 
aircraft and why and how WTC 7 collapsed; 

2. Determine why the injuries and fatalities were so high or low depending on location, 
including all technical aspects of fire protection, occupant behavior, evacuation, and 
emergency response; 

3. Determine what procedures and practices were used in the design, construction, operation, 
and maintenance of WTC 1, 2, and 7; and 

4. Identify, as specifically as possible, areas in current building and fire codes, standards, and 
practices that warrant revision. 
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NIST is a nonregulatory agency of the U.S. Department of Commerce's Technology Administration. The 
purpose of NIST investigations is to improve the safety and structural integrity of buildings in the United 
States, and the focus is on fact finding. NIST investigative teams are authorized to assess building 
performance and emergency response and evacuation procedures in the wake of any building failure that 
has resulted in substantial loss of life or that posed significant potential of substantial loss of life. NIST 
does not have the statutory authority to make findings of fault nor negligence by individuals or 
organizations. Further, no part of any report resulting from a NIST investigation into a building failure or 
from an investigation under the National Construction Safety Team Act may be used in any suit or action 
for damages arising out of any matter mentioned in such report (15 USC 281a, as amended by Public 
Law 107-231). 

Organization of the Investigation 

The National Construction Safety Team for this Investigation, appointed by the then NIST Director, 
Dr. Arden L. Bement, Jr., was led by Dr. S. Shyam Sunder. Dr. William L. Grosshandler served as 
Associate Lead Investigator, Mr. Stephen A. Cauffman served as Program Manager for Administration, 
and Mr. Harold E. Nelson served on the team as a private sector expert. The Investigation included eight 
interdependent projects whose leaders comprised the remainder of the team. A detailed description of 
each of these eight projects is available at http://wtc.nist.gov. The purpose of each project is summarized 
in Table P-l, and the key interdependencies among the projects are illustrated in Fig. P-l. 



Table P-l. Federal building and fire safety investigation of the WTC disaster. 



Technical Area and Project Leader 


Project Purpose 


Analysis of Building and Fire Codes and 
Practices; Project Leaders: Dr. H. S. Lew 
and Mr. Richard W. Bukowski 


Document and analyze the code provisions, procedures, and 
practices used in the design, construction, operation, and 
maintenance of the structural, passive fire protection, and 
emergency access and evacuation systems of WTC 1, 2, and 7. 


Baseline Structural Performance and 
Aircraft Impact Damage Analysis; Project 
Leader: Dr. Fahim H. Sadek 


Analyze the baseline performance of WTC 1 and WTC 2 under 
design, service, and abnormal loads, and aircraft impact damage on 
the structural, fire protection, and egress systems. 


Mechanical and Metallurgical Analysis of 
Structural Steel; Project Leader: Dr. Frank 
W. Gayle 


Determine and analyze the mechanical and metallurgical properties 
and quality of steel, weldments, and connections from steel 
recovered from WTC 1, 2, and 7. 


Investigation of Active Fire Protection 
Systems; Project Leader: Dr. David 
D. Evans; Dr. William Grosshandler 


Investigate the performance of the active fire protection systems in 
WTC 1 , 2, and 7 and their role in fire control, emergency response, 
and fate of occupants and responders. 


Reconstruction of Thermal and Tenability 
Environment; Project Leader: Dr. Richard 
G. Gann 


Reconstruct the time-evolving temperature, thermal environment, 
and smoke movement in WTC 1 , 2, and 7 for use in evaluating the 
structural performance of the buildings and behavior and fate of 
occupants and responders. 


Structural Fire Response and Collapse 
Analysis; Project Leaders: Dr. John 
L. Gross and Dr. Therese P. McAllister 


Analyze the response of the WTC towers to fires with and without 
aircraft damage, the response of WTC 7 in fires, the performance 
of composite steel-trussed floor systems, and determine the most 
probable structural collapse sequence for WTC 1 , 2, and 7. 


Occupant Behavior, Egress, and Emergency 
Communications; Project Leader: Mr. Jason 
D. Averill 


Analyze the behavior and fate of occupants and responders, both 
those who survived and those who did not, and the performance of 
the evacuation system. 


Emergency Response Technologies and 
Guidelines; Project Leader: Mr. J. Randall 
Lawson 


Document the activities of the emergency responders from the time 
of the terrorist attacks on WTC 1 and WTC 2 until the collapse of 
WTC 7, including practices followed and technologies used. 
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NIST WTC Investigation Projects 




Figure P-l. The eight projects in the federal building and fire safety 
investigation of the WTC disaster. 



National Construction Safety Team Advisory Committee 

The NIST Director also established an advisory committee as mandated under the National Construction 
Safety Team Act. The initial members of the committee were appointed following a public solicitation. 
These were: 



Paul Fitzgerald, Executive Vice President (retired) FM Global, National Construction Safety 
Team Advisory Committee Chair 

John Barsom, President, Barsom Consulting, Ltd. 

John Bryan, Professor Emeritus, University of Maryland 

David Collins, President, The Preview Group, Inc. 

Glenn Corbett, Professor, John Jay College of Criminal Justice 

Philip DiNenno, President, Hughes Associates, Inc. 
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• Robert Hanson, Professor Emeritus, University of Michigan 

• Charles Thornton, Co-Chairman and Managing Principal, The Thornton-Tomasetti Group, 
Inc. 

• Kathleen Tierney, Director, Natural Hazards Research and Applications Information Center, 
University of Colorado at Boulder 

• Forman Williams, Director, Center for Energy Research, University of California at San 
Diego 

This National Construction Safety Team Advisory Committee provided technical advice during the 
Investigation and commentary on drafts of the Investigation reports prior to their public release. NIST 
has benefited from the work of many people in the preparation of these reports, including the National 
Construction Safety Team Advisory Committee. The content of the reports and recommendations, 
however, are solely the responsibility of NIST. 

Public Outreach 

During the course of this Investigation, NIST held public briefings and meetings (listed in Table P-2) to 
solicit input from the public, present preliminary findings, and obtain comments on the direction and 
progress of the Investigation from the public and the Advisory Committee. 

NIST maintained a publicly accessible Web site during this Investigation at http://wtc.nist.gov. The site 
contained extensive information on the background and progress of the Investigation. 

NIST's WTC Public-Private Response Plan 

The collapse of the WTC buildings has led to broad reexamination of how tall buildings are designed, 
constructed, maintained, and used, especially with regard to major events such as fires, natural disasters, 
and terrorist attacks. Reflecting the enhanced interest in effecting necessary change, NIST, with support 
from Congress and the Administration, has put in place a program, the goal of which is to develop and 
implement the standards, technology, and practices needed for cost-effective improvements to the safety 
and security of buildings and building occupants, including evacuation, emergency response procedures, 
and threat mitigation. 

The strategy to meet this goal is a three-part NIST-led public-private response program that includes: 

• A federal building and fire safety investigation to study the most probable factors that 
contributed to post-aircraft impact collapse of the WTC towers and the 47-story WTC 7 
building, and the associated evacuation and emergency response experience. 

• A research and development (R&D) program to (a) facilitate the implementation of 
recommendations resulting from the WTC Investigation, and (b) provide the technical basis 
for cost-effective improvements to national building and fire codes, standards, and practices 
that enhance the safety of buildings, their occupants, and emergency responders. 
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Table P-2. Public meetings and briefings of the WTC Investigation. 


Date 


Location 


Principal Agenda 


June 24, 2002 


New York City, NY 


Public meeting: Public comments on the Draft Plan for the 
pending WTC Investigation. 


August 21, 2002 


Gaithersburg, MD 


Media briefing announcing the formal start of the Investigation. 


December 9, 2002 


Washington, DC 


Media briefing on release of the Public Update and NIST request 
for photographs and videos. 


April 8, 2003 


New York City, NY 


Joint public forum with Columbia University on first-person 
interviews. 


April 29-30, 2003 


Gaithersburg, MD 


NCST Advisory Committee meeting on plan for and progress on 
WTC Investigation with a public comment session. 


May 7, 2003 


New York City, NY 


Media briefing on release of May 2003 Progress Report. 


August 26-27, 2003 


Gaithersburg, MD 


NCST Advisory Committee meeting on status of the WTC 
investigation with a public comment session. 


September 17,2003 


New York City, NY 


Media and public briefing on initiation of first-person data 
collection projects. 


December 2-3, 2003 


Gaithersburg, MD 


NCST Advisory Committee meeting on status and initial results 
and release of the Public Update with a public comment session. 


February 12,2004 


New York City, NY 


Public meeting on progress and preliminary findings with public 
comments on issues to be considered in formulating final 
recommendations . 


June 18,2004 


New York City, NY 


Media/public briefing on release of June 2004 Progress Report. 


June 22-23, 2004 


Gaithersburg, MD 


NCST Advisory Committee meeting on the status of and 
preliminary findings from the WTC Investigation with a public 
comment session. 


August 24, 2004 


Northbrook, IL 


Public viewing of standard fire resistance test of WTC floor 
system at Underwriters Laboratories, Inc. 


October 19-20, 2004 


Gaithersburg, MD 


NCST Advisory Committee meeting on status and near complete 
set of preliminary findings with a public comment session. 


November 22, 2004 


Gaithersburg, MD 


NCST Advisory Committee discussion on draft annual report to 
Congress, a public comment session, and a closed session to 
discuss pre-draft recommendations for WTC Investigation. 


April 5, 2005 


New York City, NY 


Media and public briefing on release of the probable collapse 
sequence for the WTC towers and draft reports for the projects on 
codes and practices, evacuation, and emergency response. 


June 23, 2005 


New York City, NY 


Media and public briefing on release of all draft reports for the 
WTC towers and draft recommendations for public comment. 


September 12-13, 
2005 


Gaithersburg, MD 


NCST Advisory Committee meeting on disposition of public 
comments and update to draft reports for the WTC towers. 


September 13-15, 
2005 


Gaithersburg, MD 


WTC Technical Conference for stakeholders and technical 
community for dissemination of findings and recommendations 
and opportunity for public to make technical comments. 



• A dissemination and technical assistance program (DTAP) to (a) engage leaders of the 
construction and building community in ensuring timely adoption and widespread use of 
proposed changes to practices, standards, and codes resulting from the WTC Investigation 
and the R&D program, and (b) provide practical guidance and tools to better prepare facility 
owners, contractors, architects, engineers, emergency responders, and regulatory authorities 
to respond to future disasters. 

The desired outcomes are to make buildings, occupants, and first responders safer in future disaster 
events. 
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National Construction Safety Team Reports on the WTC Investigation 

A final report on the collapse of the WTC towers is being issued as NIST NCSTAR 1. A companion 
report on the collapse of WTC 7 is being issued as NIST NCSTAR 1A. The present report is one of a set 
that provides more detailed documentation of the Investigation findings and the means by which these 
technical results were achieved. As such, it is part of the archival record of this Investigation. The titles 
of the full set of Investigation publications are: 

NIST (National Institute of Standards and Technology). 2005. Federal Building and Fire Safety 
Investigation of the World Trade Center Disaster: Final Report on the Collapse of the World Trade 
Center Towers. NIST NCSTAR 1. Gaithersburg, MD, September. 

NIST (National Institute of Standards and Technology). 2008. Federal Building and Fire Safety 
Investigation of the World Trade Center Disaster: Final Report on the Collapse of World Trade Center 7. 
NIST NCSTAR 1A. Gaithersburg, MD, November. 

Lew, H. S., R. W. Bukowski, and N. J. Carino. 2005. Federal Building and Fire Safety Investigation of 
the World Trade Center Disaster: Design, Construction, and Maintenance of Structural and Life Safety 
Systems. NIST NCSTAR 1-1. National Institute of Standards and Technology. Gaithersburg, MD, 
September. 

Fanella, D. A., A. T. Derecho, and S. K. Ghosh. 2005. Federal Building and Fire Safety 
Investigation of the World Trade Center Disaster: Design and Construction of Structural Systems. 
NIST NCSTAR 1-1 A. National Institute of Standards and Technology. Gaithersburg, MD, 
September. 

Ghosh, S. K., and X. Liang. 2005. Federal Building and Fire Safety Investigation of the World 
Trade Center Disaster: Comparison of Building Code Structural Requirements. NIST 
NCSTAR 1-1B. National Institute of Standards and Technology. Gaithersburg, MD, September. 

Fanella, D. A., A. T. Derecho, and S. K. Ghosh. 2005. Federal Building and Fire Safety 
Investigation of the World Trade Center Disaster: Maintenance and Modifications to Structural 
Systems. NIST NCSTAR 1-lC. National Institute of Standards and Technology. Gaithersburg, 
MD, September. 

Grill, R. A., and D. A. Johnson. 2005. Federal Building and Fire Safety Investigation of the World 
Trade Center Disaster: Fire Protection and Life Safety Provisions Applied to the Design and 
Construction of World Trade Center 1, 2, and 7 and Post-Construction Provisions Applied after 
Occupancy. NIST NCSTAR 1-1D. National Institute of Standards and Technology. Gaithersburg, 
MD, September. 

Razza, J. C, and R. A. Grill. 2005. Federal Building and Fire Safety Investigation of the World 
Trade Center Disaster: Comparison of Codes, Standards, and Practices in Use at the Time of the 
Design and Construction of World Trade Center 1, 2, and 7. NIST NCSTAR 1-1E. National 
Institute of Standards and Technology. Gaithersburg, MD, September. 

Grill, R. A., D. A. Johnson, and D. A. Fanella. 2005. Federal Building and Fire Safety 
Investigation of the World Trade Center Disaster: Comparison of the 1968 and Current (2003) New 
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York City Building Code Provisions. NIST NCSTAR 1-1F. National Institute of Standards and 
Technology. Gaithersburg, MD, September. 

Grill, R. A., and D. A. Johnson. 2005. Federal Building and Fire Safety Investigation of the World 
Trade Center Disaster: Amendments to the Fire Protection and Life Safety Provisions of the New 
York City Building Code by Local Laws Adopted While World Trade Center 1, 2, and 7 Were in 
Use. NIST NCSTAR 1-1G. National Institute of Standards and Technology. Gaithersburg, MD, 
September. 

Grill, R. A., and D. A. Johnson. 2005. Federal Building and Fire Safety Investigation of the World 
Trade Center Disaster: Post-Construction Modifications to Fire Protection and Life Safety Systems 
of World Trade Center 1 and 2. NIST NCSTAR 1-1H. National Institute of Standards and 
Technology. Gaithersburg, MD, September. 

Grill, R. A., D. A. Johnson, and D. A. Fanella. 2005. Federal Building and Fire Safety Investigation 
of the World Trade Center Disaster: Post-Construction Modifications to Fire Protection, Life 
Safety, and Structural Systems of World Trade Center 7. NIST NCSTAR 1-11. National Institute of 
Standards and Technology. Gaithersburg, MD, September. 

Grill, R. A., and D. A. Johnson. 2005. Federal Building and Fire Safety Investigation of the World 
Trade Center Disaster: Design, Installation, and Operation of Fuel System for Emergency Power in 
World Trade Center 7. NIST NCSTAR 1-1 J. National Institute of Standards and Technology. 
Gaithersburg, MD, September. 

Sadek, F. 2005. Federal Building and Fire Safety Investigation of the World Trade Center Disaster: 
Baseline Structural Performance and Aircraft Impact Damage Analysis of the World Trade Center 
Towers. NIST NCSTAR 1-2. National Institute of Standards and Technology. Gaithersburg, MD, 
September. 

Faschan, W. J., and R. B. Garlock. 2005. Federal Building and Fire Safety Investigation of the 
World Trade Center Disaster: Reference Structural Models and Baseline Performance Analysis of 
the World Trade Center Towers. NIST NCSTAR 1-2A. National Institute of Standards and 
Technology. Gaithersburg, MD, September. 

Kirkpatrick, S. W., R. T. Bocchieri, F. Sadek, R. A. MacNeill, S. Holmes, B. D. Peterson, 
R. W. Cilke, C. Navarro. 2005. Federal Building and Fire Safety Investigation of the World Trade 
Center Disaster: Analysis of Aircraft Impacts into the World Trade Center Towers, NIST 
NCSTAR 1-2B. National Institute of Standards and Technology. Gaithersburg, MD, September. 

Gayle, F. W., R. J. Fields, W. E. Luecke, S. W. Banovic, T. Foecke, C. N. McCowan, T. A. Siewert, and 
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Executive Summary 



Survey administration involved interviews with survivors of the World Trade Center (WTC) attacks using 
three methods, telephone interviews, face-to-face interviews, and focus groups, and the delivery of the 
resultant data. This report documents the methods and outcomes of these interviews. Analyses of the 
resultant data were conducted by National Institute of Standards and Technology (NIST) investigators. 
The interviews were carried out by a data collection team consisting of NuStats Partners, LP, Austin, 
Texas, as the prime contractor, with the assistance of subcontractors - NuStats DataSource, San Marcos, 
Texas, to conduct telephone interviews; GeoStats, Atlanta, Georgia, to provide data collection devices 
and programming services for the face-to-face interviews; and Diversity Services, New York City, to 
recruit surveyors and provide office space. 

The goals of the NIST World Trade Center investigation were to investigate the building construction, the 
materials used, and the technical conditions that contributed to the outcome of the WTC disaster. The 
results of the investigation were meant to serve as the basis for improvements in the way buildings are 
designed, constructed, maintained, and used; improved tools, guidance for industry and safety officials; 
revisions to codes, standards, and practices; and improved public safety. 

This research was conducted in compliance with NIST requirements regarding the protection of human 
subjects. The Office of Human Research Protections (OHRP) in the Department of Health and Human 
Services (DHHS) registered Federalwide Assurance (FWA) numbers for both NuStats Partners, LP and 
NuStats DataSource, LP. For both of these FWA entities, the Institutional Review Board (IRB) 
designated for review was Essex IRB, Lebanon, New Jersey 

NuStats prepared a comprehensive IRB package for all data collection protocols, and submitted the 
package to Essex IRB. Subsequent to NIST review, Essex approved a revised IRB package. An 
important partner in the provision of human subjects protection was Jamie Abelson, a senior research 
associate affiliated with the University of Michigan Institute for Social Research. Based on her vast 
experience in dealing with sensitive interviewing situations, she provided hands-on mental health support, 
as needed, to both surveyors and respondents. NIST further worked with NuStats to ensure that the data 
collection met the requirements of the Paperwork Reduction Act and was awarded an Office of 
Management and Budget (OMB) approval number (0693-0044). 

Three data collection methodologies were employed for this investigation, including: (1) a telephone 
interview of 803 occupants of the WTC towers, utilizing a scientific probability sample of surviving 
occupants was used to identify survey participants; (2) a qualitative study of 220 face-to-face interviews 
of occupants and family members of victims, utilizing a snowball quota sample of key survivor types and 
the toll-free hotline were used to identify potential participants; and (3) six focus groups of occupants, 
relying on a snowball quota sampling methodology for recruiting key survivor participants. 
The primary goal of the telephone interview was to provide quantitative information on occupant 
behavior during the evacuation experience and capture unique investigative observations, particular to the 
events at the WTC on September 11, 2001. This data collection activity also sought to estimate the 
occupant population in the WTC at the time of the attacks on September 11. The mode of data collection 
was computer-assisted telephone interviewing (CATI). 
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The interview was designed to measure five primary content areas, presented to the respondent in the 
following order: 

1. Preparedness and Training - These questions collected information on the respondents' 
degree of evacuation or emergency training pertinent to their September 1 1 experience. The 
questions also addressed familiarity with the WTC buildings based on length employment at 
the location and also whether the respondents were at the WTC in 1 993 during that prior 
bombing incident. 

2. Initial September 1 1 Experience - These questions captured data on respondents' initial 
awareness that something had happened to the building - where they were, what they were 
doing, how they first became aware, sources of information about the event (both in terms of 
other persons and also in terms of their physical environment). 

3. Interim September 1 1 Experience - These questions addressed the span between the time 
respondents first became aware of the WTC incident and when they began evacuating - data 
items included their social environment, physical environment, information sources, and 
factors influencing their decision to evacuate. 

4. Evacuation Experience on September 1 1 - These questions captured information on the 
respondents' egress from the building, including the factors that facilitated their egress and 
those that hindered their egress. 

5. Respondent Characteristics - Information collected included age, gender, language, and 
mobility impairments. 

The objective of the face-to-face interviews was to gather first-hand accounts and observations of the 
activities and events inside the WTC buildings on the morning of September 11. The qualitative nature of 
the face-to-face interviews identified heretofore unknown information, evaluated technical hypotheses, 
and explored conscious and subconscious motivations for occupant and responder behaviors, while 
allowing for qualitative comparisons to the telephone interview data. The face-to-face interview 
methodology was a synthesis of the Behavioral Sequence Interview Technique originally developed by 
Keating and Loftus, and the Cognitive Interviewing Methods, originally developed by Fisher and 
Gieselman. 

The following special groups of building occupants were targeted in the face-to-face interviews. NIST 
and the expert panel selected these special groups for the face-to-face interviews because these groups 
were believed to hold unique pieces of information to add to the investigation. They either belong to 
special subpopulation groups or were in locations during the evacuation that enabled them to experience 
distinctive situations. Because these respondents were expected to represent relatively rare types of 
occupants or other targeted groups, a snowball sampling approach was used to locate respondents. 
Snowball sampling relies on referrals from initial subjects to generate additional subjects. The selection 
of this approach, a non-probability design, eliminated the potential for generalizing to the population of 
inference (i.e., occupants of WTC towers on September 1 1), but the design provided access to rare, hard- 
to-find persons with unique information to provide to the investigation. The special groups targeted in the 
face-to-face interviews were: (a) people near the floors of impact; (b) people in the lobby who witnessed 
fireballs; (c) families of victims who talked to victim during disaster; (d) Occupants from WTC 7; (e) 
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persons with building responsibilities; (f) people who used or were trapped in elevators; (g) people with 
mobility challenges; and (h) random evacuees in WTC 1 and WTC 2. 

The face-to-face interview instrument was drafted as a computer-assisted personal interview (CAP I) 
application that would operate on Pen Tablet PCs. With CAPI technology, high quality qualitative data 
were electronically coded, organized, and stored directly in an electronic format, with all relevant error 
checking performed automatically. Not only was this application more appropriate for the prescribed data 
collection protocol, this process simultaneously eliminated unnecessary data entry costs and associated 
key entry errors that typically occur in paper-based field surveys. Both the Pen Tablet PCs and the 
application were provided by GeoStats. 

The goal of the focus groups was to elicit group representations of specific events or themes. Distinct 
categories of persons were provided by NIST based on preliminary analyses of CATI survey and face-to- 
face interview data. These groups were prioritized based on expectations of sample availability and 
information value. 

Six focus groups were completed. The groups were: 

1 . Random survivors from WTC 1 ; 

2. Random survivors from WTC 2; 

3. Mobility challenged individuals; 

4. Persons having building responsibility; 

5. Persons near the floors of impact; and 

6. Floor wardens. 

The venues for the focus groups were changed from Manhattan to outside of Manhattan depending on the 
preferences of focus group respondents. Among respondents who were willing to participate in the focus 
group, several no longer traveled into Manhattan. Of the persons who participated in the groups, all were 
active in the discussion and willing to share their experiences, thoughts, and recommendations. 

NuStats was responsible for delivery of databases containing information resulting from the interviews 
with survivors. These databases comprised various formats, specific to the type of interview conducted. 

In addition to delivering databases, NuStats also provided database support services. These services 
included locating a subcontractor to provide qualitative analysis support for the face-to-face interviews 
and providing statistical expertise to support model development using the CATI data. 
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Chapter 1 

Introduction 



Survey administration involved interviews with survivors of the World Trade Center (WTC) attacks using 
three methods, telephone interviews, face-to-face interviews, and focus groups, and the delivery of the 
resultant data. This report documents the methods and outcomes of these interviews. (Appendix A 
contains the work program that guided the work conducted herein. 1 ) Analyses of the resultant data were 
conducted by National Institute of Standards and Technology (NIST) investigators. The interviews were 
carried out by a data collection team consisting of NuStats Partners, LP, Austin, Texas, as the prime 
contractor, with the assistance of subcontractors — NuStats DataSource, San Marcos, Texas, to conduct 
telephone interviews; GeoStats, Atlanta, Georgia, to provide data collection devices and programming 
services for the face-to-face interviews; and Diversity Services, New York City, to recruit surveyors and 
provide office space. 

1.1 OBJECTIVE 

The goals of the NIST WTC investigation were to investigate the building construction, the materials 
used, and the technical conditions that contributed to the outcome of the WTC disaster. The results of the 
investigation were meant to serve as the basis for improvements in the way buildings are designed, 
constructed, maintained, and used; improved tools, guidance for industry and safety officials; revisions to 
codes, standards, and practices; and improved public safety. While multiple sources and types of data 
were sought to meet the needs of the NIST investigation, the scope of services under this contract 
provided important first-hand accounts of events and experiences on September 11, 2001, through 
telephone interviews, face-to-face interviews, and focus groups. This data collection effort evaluated the 
role of occupant behavior and evacuation technologies and practices for tall buildings, including decision- 
making and situation awareness, time-constrained evacuation strategies, communications, role of floor 
wardens and fire safety directors, and issues concerning people with disabilities. Additionally, 
observations of fire and smoke conditions and/or structural damage from within the building were sought. 
Families of the victims, or others who communicated with loves ones inside the towers before collapse, 
were also interviewed primarily to determine the nature of the environment above the floors of impact. 

1.2 PROTECTION OF HUMAN SUBJECTS 

This research was conducted in compliance with NIST requirements regarding the protection of human 
subjects. The Office of Human Research Protections (OHRP) in the Department of Health and Human 
Services (DHHS) registered Federalwide Assurance (FWA) numbers for both NuStats Partners, LP 
(FWA00000562) and NuStats DataSource, LP (FWA00005084). For both of these FWA entities, the 
Institutional Review Board (IRB) designated for review was Essex IRB, Lebanon, New Jersey. Essex 



Whereas the work program provided in Appendix A refers to interviews with first responders, subsequent agreements between 
NIST and the City of New York led to NIST personnel conducting these interviews. This level of effort by the contractor was 
replaced by the provision of other services. 
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IRB is registered with OHRP and is in good standing with OHRP. Essex's IRB identification number is 
IRB00001742. 

NuStats prepared a comprehensive IRB package for all data collection protocols, and submitted the 
package to Essex IRB in September 2003. Essex approved this package on September 26. In 
October 2003, NuStats revised the IRB package based on NIST legal review, and then NuStats submitted 
the revised IRB package to Essex IRB for review. On November 10, Essex approved this revised IRB 
package. Based on subsequent issues raised by NIST legal, NuStats revised the IRB package once again, 
and submitted this revised package to Essex IRB for review. On November 17, Essex approved this 
revised IRB package. 

An important partner in the provision of human subjects protection was Jamie Abelson, a senior research 
associate affiliated with the University of Michigan Institute for Social Research. Based on her vast 
experience in dealing with sensitive interviewing situations, she provided hands-on mental health support, 
as needed, to both surveyors and respondents. 

1.3 PAPERWORK REDUCTION ACT 

The Paperwork Reduction Act of 1995, effective October 1, 1995, requires that Office of Management 
and Budget (OMB) approve each collection of information by a Federal agency before it can be 
implemented. NIST prepared a package of materials for the telephone data collection. NuStats reviewed 
these OMB materials to ensure consistency with survey and sample design parameters and with OMB 
reporting requirements. In September 2003, NIST was granted OMB approval number 0693-0044. 

1.4 TOLL-FREE HOTLINE 

At NIST's request, NuStats set up a toll-free (1-800) number with a staff of telephone operators. This line 
was used to capture requests for information about the NIST investigation, as well as to screen potential 
respondents for the telephone, face-to-face, or focus group interviews. A screener interview was 
conducted with hotline callers to cull those who fit the necessary profile for unique groups to be 
interviewed, primarily in the face-to-face interviewing effort. Summaries of hotline contacts by day were 
prepared for NIST review. 
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Three data collection methodologies were employed for this investigation, as presented in the bullets 
below. 



• 



• 



Telephone Survey: a telephone interview survey of 800 occupants of the World Trade 
Center (WTC) towers; a scientific probability sample of surviving occupants was used to 
identify survey participants. 

In-Depth Personal Interviews: a qualitative study of approximately 220 face-to-face 
interviews of occupants and family members of victims; a snowball quota sample of key 
survivor types and the toll-free hotline were used to identify potential participants. 

Focus Groups: five focus groups of occupants were conducted; this method also relied on a 
snowball quota sampling methodology for recruiting key survivor participants. 

Prior to the NuStats contract, National Institute of Standards and Technology (NIST) staff and contractor 
experts selected this multi-method approach for several reasons. 2 First, multiple methods would increase 
confidence in the conclusions and findings when more than one method arrived at the same conclusions, a 
process known as triangulation. Second, the multiple objectives of the investigation mandated 
complementary approaches to accomplish all the goals. In other words with only one method, it would be 
difficult to establish a scientific foundation for general findings while also broadly investigating and 
establishing new facts and discovering unique events. Finally, concerns associated with the time latency 
since September 1 1, 2001, suggested the use of different approaches and techniques in order to increase 
memory recall and accuracy. 3 The original methodology paper is included in Appendix B. 

2.1 TELEPHONE SURVEY 4 

The primary goal of the telephone interview was to provide quantitative information on occupant 
behavior during the evacuation experience and capture unique investigative observations, particular to the 
events at the WTC on September 11, 2001. This data collection activity also sought to estimate the 



The NIST project staff primarily involved in this effort included Jason Averill, Erica Kuligowski, Randy Lawson, Richard 
Peacock, and Paul Reneke. Contractor experts included Dr. Norman Groner, John Jay College of Criminal Justice in New 
York City, Dr. Dennis Mileti, Director of the National Hazards Research and Applications Information Center within the 
Institute of Behavioral Science at the University of Colorado at Boulder, and Dr. Guylene Proulx, Research Officer from the 
Institute for Research in Construction at the National Research Council of Canada. 

From "Data Collection Methodology for World Trade Center Evaluation and Emergency Response: Telephone Interviews, 
Face-to-Face Interviews, Focus Groups and Population Sampling." Jason Averill, Erica Kuligowski, Randy Lawson, Richard 
Peacock, and Paul Reneke, National Institute of Standards and Technology, Gaithersburg, MD and Norman Groner, Dennis 
Mileti, and Guylene Proulx. 

The information in this section relies on the content of NIST NCSTAR 1-7, prepared by NIST from information provided by 
NuStats. 
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occupant population in the WTC at the time of the attacks on September 1 1 . The mode of data collection 
was computer-assisted telephone interviewing (CATI). 

The contract required the collection of 800 CATI interviews with persons occupying either of the two 
WTC towers (WTC 1, WTC 2) at the time of the terrorist attacks on September 11, 2001. Attempts were 
made to equally divide the respondents between WTC 1 and WTC 2 occupants (i.e., n = 400 occupant 
interviews from each tower). A proportionate stratified sample design drew samples of survivors within 
each of the WTC buildings. This maximized the precision associated with contrasts between the two 
building-specific groups of occupants, and provided a nominal level of precision for statistically 
analyzing the characteristics of survivors from each building. Robert Santos, NuStats, developed this 
sampling approach and the design parameters for it. 

2.1.1 Sampling Approach 

The sampling frame (i.e., the list from which the sample was drawn) consisted of the names of occupants 
from badge lists of WTC 1 and WTC 2. All occupants of the WTC were required to provide personal 
data in support of issuing badges to clear through the security station at the entrance of each tower. The 
badge lists were provided to NIST by the Port Authority of New York /New Jersey. For sampling 
purposes, the lists provided name, floor of occupancy, employer, and social security number. 
Unfortunately, contact information was not provided, which added unanticipated complexity to the CATI 
survey task. In addition, the badge list contains September 1 1 occupants, occupants who were absent on 
the day of the attacks, decedents, and former occupants. 

As mentioned, a proportionate stratified sample design was developed for this survey effort. The sample 
was stratified on two variables: (1) Building Category (WTC 1 vs. WTC 2); and (2) Floor of Occupancy 
(three groupings). This resulted in six primary strata as defined below: 

WTC 1 - floors 0-43 

WTC 1 - floors 44-76 

WTC 1 - floors 77 and above 

WTC 2 - floors 0-43 

WTC 2 - floors 44-76 

WTC 2 - floors 77 and above. 

Secondary stratification involved two categories of Tenant Square Footage of a Floor: (1) occupants 
employed by tenants occupying up to 40 percent of a floor's square footage, and (2) occupants employed 
by tenants occupying more than 40 percent of a floor's square footage. 

Given that the sampling frame contained both eligible and ineligible respondents, an intensive screening 
effort was needed to identify "eligible" badge list members - namely, those who were inside WTC 1 or 
WTC 2 during the attacks. Moreover, the absence of telephone numbers for the badge holders on the list 
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necessitated a tracking/locating effort. The primary tracking mechanism was to search public databases 
using the LexisNexis program. 5 This necessitated a large sample to generate the 800 desired interviews. 

The number of occupant selections drawn into the sample was contingent on four key design parameters: 

• the percentage of individuals from badge listings for whom a working telephone number can 
be found (Initial estimate: 80 percent tracking success) 

• the percentage of badge listings that correspond to a surviving WTC 1, WTC 2 occupant on 
September 11 (Initial estimate: 14 percent) 

• the cooperation rate for screening the occupants (Initial estimate: 65 percent) 

• the interview response rate among 9/1 1 survivors (Initial estimate: 50 percent). 

Given these design parameters, a total sample of approximately 22,735 persons from the badge list 
needed to be tracked and located to generate the desire 800 completed interviews. A reserve sample of 
20 percent (or about n=4,550) was added in the event additional sample was needed due to unanticipated 
circumstance (e.g., the eligibility rate is lower than anticipated). The reserve was held "in reserve" while 
the main sample was worked. Working the main sample allowed preliminary estimates of all design 
parameters to be monitored so that an informed decision could be made on the necessity of releasing 
none, some or all of the reserve. Equal samples were drawn from the collections of badge holders in 
WTC 1 and WTC 2. That is, half of the 22,735 selections were allocated to the WTC 1 sample, and the 
other half went to the WTC 2 sample. The reserve samples were similarly drawn. 

2.1.2 Instrument Development 

NuStats developed the telephone instrument in close association with NIST project staff and the expert 
panel members. A one-day meeting of these individuals was held at NIST in July 2003 to review an 
instrument draft. Subsequent to this meeting, NuStats prepared a conceptual map of the instrument to 
prioritize measurement areas. Subsequent in-person and teleconference meetings relied on this 
conceptual map to refine the measurement areas and finalize the draft instrument. The draft instrument 
was provided to Dr. Jon Krosnick, a cognitive psychologist and nationally recognized expert in 
questionnaire design, for review in terms of controlling for non-response and measurement errors and to 
Jamie Abelson for review for human subjects and surveyor training implications. 6 After their review and 
subsequent final reviews by NIST project staff and the expert panel, the CATI instrument was finalized. 

The interview was designed to measure five primary content areas, presented to the respondent in the 
following order: 

• Preparedness and Training - These questions collected information on the respondents' 
degree of evacuation or emergency training pertinent to their September experience. The 



LexisNexis is a web-based search product that enables subscribers to search nationwide databases to locate information on 



individuals, verify addresses, and find telephone numbers. 

Dr. Jon Krosnick, Ohio State University Departments of Psychology and Political Science; Jamie Abelson, University of 
Michigan Institute for Social Research. 
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questions also addressed familiarity with the WTC buildings based on length of employment 
at the location and also whether the respondents were at the WTC in 1993 during that prior 
bombing incident. 

• Initial September 1 1 Experience - These questions captured data on respondents' initial 
awareness that something had happened to the building - where they were, what they were 
doing, how they first became aware, sources of information about the event (both in terms of 
other persons and also in terms of their physical environment). 

• Interim September 1 1 Experience - These questions addressed the span between the time 
respondents first became aware of the WTC incident and when they began evacuating - data 
items included their social environment, physical environment, information sources, and 
factors influencing their decision to evacuate. 

• Evacuation Experience on September 1 1 - These questions captured information on the 
respondents' egress from the building, including the factors that facilitated their egress and 
those that hindered their egress. 



• 



Respondent Characteristics - Information collected included age, gender, language, and 
mobility impairments. 

A pilot test of the CATI instrument was conducted in November 2003, directed by Heather Contrino, 
NuStats. Eleven interviews were completed. In the pilot, all aspects of the data collection plan were 
evaluated, including, instrument flow, respondent comprehension, interviewer administration, 
identification of questions needing probes or clarifications, data coding and processing, and review of 
survey data. Only very minor modifications to the instrument were required based on pilot outcomes. 

2.1.3 CATI Interviewing 

Highly experienced interviewers at DataSource conducted the telephone interviews using a computer 
program which provided questions and answer categories. Prior to calling, subjects received an advance 
letter that outlined the scope and purpose of the investigation, the purpose of the interview, and the 
telephone call that came several days later. A full informed consent statement appeared in the letter, as 
well. When interviewers reached the respondents by telephone, respondents were provided a description 
of the survey, the confidentiality of responses, the length of the interview, and the voluntary nature of 
participation. Respondents were then asked if they wished to participate, thereby obtaining oral informed 
consent. 

Interviewer training was held on December 1 and 2, 2003, and actual dialing of the sample began on 
December 3. Interviewing continued until January 23, 2004, with a short hiatus during the December 
holidays. The actual productivity of the CATI interviewing was lower than initial design parameters. 
Interviewing productivity rates are provided in Chapter 3 of this report. In December 2003, pre- 
notification letters were inadvertently mailed to approximately 47 decedent persons. This event occurred 
because the badge list was not properly flagged for decedents. Once the error surfaced, the database of 
respondents (i.e., the badge list) was manually matched and checked against known decedents to ensure 
that all potential decedents in the sample were flagged and removed from the active sample. Letters of 
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apology were mailed to decedent households. In total, 803 CATI interviews were completed. Details on 
the outcomes of the CATI interviewing are presented in Chapter 3. 

2.2 FACE-TO-FACE INTERVIEWS 

The objective of the face-to-face interviews was to gather first-hand accounts and observations of the 
activities and events inside the WTC buildings on the morning of September 11. The qualitative nature of 
the face-to-face interviews identified heretofore unknown information, evaluated technical hypotheses, 
and explored conscious and subconscious motivations for occupant and responder behaviors, while 
allowing for qualitative comparisons to the telephone interview data. 

The contract required that the face-to-face interviews be conducted via a relative new methodology, that 
was a synthesis of the Behavioral Sequence Interview Technique (BSIT) originally developed by Keating 
and Loftus, 7 and the Cognitive Interviewing Methods (CIM), originally developed by Fisher 8 and 
Gieselman. 9 Both approaches begin by allowing the respondent to retell an unimpeded account without 
interruption from the interviewer, and both initially employ a chronological retelling of information. The 
BSIT was designed to yield a database of qualitative information that could be subjected to systematic 
analysis and consolidation, while CIM was designed to facilitate investigative interviews. Since the WTC 
investigation was pursuing both goals (i.e., creation of a database of evacuation-related behaviors and an 
investigatory attempt to capture information relevant to outcomes), the face-to-face methodology was 
designed to combine these two approaches. 

2.2.1 Sampling Approach 

The following special groups of building occupants were targeted in the face-to-face interviews. NIST 
and the expert panel selected these special groups for the face-to-face interviews because these groups 
were believed to hold unique pieces of information to add to the investigation. They either belong to 
special subpopulation groups or were in locations during the evacuation that enabled them to experience 
distinctive situations. Because these respondents were expected to represent relatively rare types of 
occupants or other targeted groups, a snowball sampling approach was used to locate respondents. 
Snowball sampling relies on referrals from initial subjects to generate additional subjects. In the case, 
referrals came from NIST public announcements (at press conferences in New York City and on the NIST 
web site), from CATI interview respondents, and from associated parties (i.e., NIST, Port Authority of 
New York/ New Jersey). The selection of this approach, a non-probability design, eliminated the 
potential for generalizing to the population of inference (i.e., occupants of WTC towers on September 1 1) 
but the design provided access to rare, hard-to-find persons with unique information to provide to the 
investigation. The special groups targeted in the face-to-face interviews are described as follows. 
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Occupants Subgroup A: People near the Floors of Impact 

• 50 people from floors 90 - 94 in WTC 1 at the time of WTC 1 impact. 

• 50 people from floors 75 - 1 10 in WTC 2 at the time of WTC 2 impact. 

Rationale for inclusion: These people witnessed fire, smoke, structural/ system damage, heard 
PA announcements, and walked the greatest distance to evacuate the buildings. 

Occupants Subgroup B: People in the Lobby who Witnessed Fireballs 

• 10 people who witnessed fireballs. 

Rationale for inclusion: Estimating the size and source of the fireball to identify the path the fuel 
traveled, how much fuel was consumed, and how much damage to the lobby resulted. 

Occupants Subgroup C: Families of Victims who Talked to Victim during Disaster. 

• 30 family members or others. 

Rationale for inclusion: These people who communicated with victims may be the sole source of 
"first-hand" information from above the floors of impact. 

Occupants Subgroup D: People in WTC Building 7 

• 15 people who evacuated 

• 5 people with some level of decision-making authority over the evacuation. 

Rationale for inclusion: Determine the total evacuation time for Building 7, total number of 
people evacuated, density of people in the stairwells, and evaluate when fires may have started. 

Occupants Subgroup E: Persons with Building Responsibility 

• 50 floor wardens, fire safety directors, or building management staff. 

Rationale for inclusion: Involved in decision-making process on September 1 1 about who and 
how to evacuate the buildings. Know who had what information, when they had it, and what they 
did with it. 

Occupants Subgroup F: People who Used or were Trapped in Elevators 

• 25 people trapped in elevators or used elevators after their building was hit. 

Rationale for inclusion: Estimating the number of people who may have been killed inside 
elevators and what the failure mechanism of the elevator system was. 
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Occupants Subgroup G: People with Disabilities 

• 20 people who experienced a mobility impairment prior to, or during, their evacuation - 
anything from a disability under the Americans with Disabilities Act (ADA) to pregnancy, 
overweight, age, etc. 

Rationale for inclusion: Document the challenges faced by people with disabilities in order to 
better address those needs in future building incidents as well as identify pro-social behaviors, 
rescue, or devices that contributed to their ultimate evacuation success. 

Occupants Subgroup H: Random Evacuee 

• 25 people from each of the lower and middle thirds of WTC 1 

• 25 people from each of the lower and middle thirds of WTC 2 (that is, not from the floors 
near the impact). 4 zones x 25 people =100 people. 

Rationale for inclusion: Document typical evacuation experience. Ensure unanticipated events 
or experiences were captured. 

2.2.2 instrument Development 

NuStats developed the face-to-face instrument in close association with NIST project staff and the NIST 
expert panel members. However, after this draft was prepared it became apparent that the prescribed data 
collection protocol for the face-to-face interviews would determine the instrument structure and content. 
Thus, the instrument was drafted as a computer-assisted personal interview (CAP I) application that would 
operate on Pen Tablet PCs rather than as a paper document (see Appendix A, Section A. 3.2. 1). With 
CAPI technology, high quality qualitative data were electronically coded, organized, and stored directly 
in an electronic format, with all relevant error checking performed automatically. Not only was this 
application more appropriate for the prescribed data collection protocol, this process simultaneously 
eliminated unnecessary data entry costs and associated key entry errors that typically occur in paper-based 
field surveys. Both the Pen Tablet PCs and the application were provided by GeoStats. 

The CAPI application captured information from participants in four (4) steps. 

Step 1: Uninterrupted Narrative Account 

The participants were asked to relate their experience on September 1 1 from the time that they first knew 
something was wrong to the time when they exited the building. Researchers and practitioners involved 
with cognitive interviewing believe that starting the face-to-face interviews in this manner both improves 
recall and helps build rapport between the participant and the interviewer. The surveyors were trained to 
listen intently and to take notes during this stage. 

Step 2: Collaborative, Structured Action-Cue-Goal Account 

After the participant completed their story, the respondent was asked to re-visit their experience. 
Interviewers worked cooperatively with participants to record entries into a CAPI table (see Table 2-1). 
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During this collaboration, interviewers recorded the information, guided the process, and asked questions 
to clarify and define the content. In the structured table, each row of the table was a triad comprised of an 
action, a cue, and a goal. 

Table 2-1. Example tabular face-to-face interview data entry. 



Action 


Cue 


Goal 


I stumbled in the dark towards 
where I thought the voice came 


I heard someone yell "I've found 
a clear path" 


So that I could find a way to 

escape 


So I called out to whoever yelled, 
"I'm near the reception area. 
Where are you?" 


My path was blocked by debris 


To try to get a better idea of 
where the person was 



• Action. An action was something that the participant did as a reaction to a cue. Actions 
should be specific enough for a reader to visualize. 

• Cue. A cue was something that starts an action. Cues can be external (something one sees, 
hears, smells, feels) or something internal (a decision, a memory). 

• Goal. Goals were what the participant was trying to accomplish by taking the action. 

Step 3: Review and Edit Account of September 11 Experience 

After the table was completed collaboratively, the interviewer asked the participant to review what had 
been recorded to ensure that it accurately captured the participant's experience. 

Step 4: Topic Cards 

After completing the negotiated, structured narrative account, the interviewer showed the participant 
"Topics" cards and asked the participant to identify those topics that were relevant to his/her experience. 
The topics were intended to capture specific information of particular value to the investigation. If a topic 
was relevant, the interviewer recorded the open-ended statements of the participants in text boxes that 
were associated with the relevant A-C-G row in the structured table. The interviewing application was 
programmed to enable this to be easily done. 

On September 9 and 10, 2003 in New York City, NuStats interviewed three WTC building occupants 
with the purpose of beta testing the face-to-face CAPI application. These beta test interviews were 
conducted by Dr. Johanna Zmud and Ms. Delia Santos. Results of the beta test were compiled and 
delivered to NIST. The three individuals were paid as consultants to NuStats because IRB approval had 
not been obtained at the time. The interviews ranged from 1 to 3 hours. Respondents reacted quite 
positively to the "cue", "action", and "reason" (C-A-R) protocol. The act of being interviewed on their 
September 1 1 experiences was an emotionally trying activity for the respondents. Respondents arrived at 
the interview site emotionally stressed. Their unimpeded narratives were spoken with emotion and 
sometimes their eyes teared. The C-A-R protocol, however, seemed to objectify the experience for them. 
Thus, the negotiated description of their experience was done with much less emotion than the unimpeded 
narrative and greater detail resulted. Generally, the unimpeded narratives took approximately one-quarter 
the time of the negotiated C-A-R activity. The respondents appeared genuinely to be engaged in working 
with the interviewer to complete the C-A-R table. Respondents also took ownership of the content of the 
C-A-R table. The beta test identified some mechanical problems with the application and also some 
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conceptual issues pertaining to clarifying C-A-R. Revisions to the application and to the conceptual 
underpinnings of the CAPI application were made. 

A pilot test of the revised CAPI application was conducted on November 5 and 6 in NYC. NuStats 
NY-based coordinator (Ms. Maria Elena Ramos) recruited five respondent consultants for these pilot 
interviews. The interviews were conducted by Dr. Johanna Zmud and Delia Santos. The pilot revealed 
that the modifications to the application enhanced its user-interface. 

2.2.3 Face-to-Face Interviews with Occupants 

The face-to-face interviews were conducted by surveyors recruited by a subcontractor, Diversity Services 
in New York City. Surveyors were sought who had some social work, interviewing, or client-interaction 
experience. Training of the surveyors was held over a 3-day period in early November 2003 in New York 
City. Surveyors were required to attend all three days of training. Surveyors were responsible for setting 
up the interview appointments, completing the interviews, and uploading interview files to the secure 
NuStats file transfer protocol (FTP) site for data editing and processing. 

Through Diversity Services, NuStats trained 12 surveyors of which ten continued interviewing through 
mid- February 2004. A skeleton crew of five surveyors continued interviewing through the end of 
April 2004 for completing face-to-face interviews with really hard-to-find special groups, to recruit focus 
group participants, and to conduct coding of 9-1-1 tapes and transcripts of interviews with first responders 
as a special assignment from NIST. 

Respondents for the face-to-face interviews were identified through three avenues: (1) NIST, (2) Hotline, 
(3) CATI survey. From these sources, persons were identified with whom a DataSource interviewer 
would conduct a telephone screener interview. Respondents who successfully completed a screener 
interview were coded based on their affiliation with one or more of the target groups and placed into the 
sampling frame. The day-to-day management of this sample was critical to completing interviews with 
this hard-to-reach population. The sample file was housed on the NuStats' FTP site. The NYC-based 
coordinator downloaded sample information daily and distributed the new sample to surveyors. 
Surveyors scheduled interviews and emailed the information back to the coordinator, who then posted it 
on the NuStats' FTP site so that the information on potential and completed interviews was kept in real- 
time. The information on completed, missed, or rescheduled interviews was tracked at NuStats and a 
report was prepared three-times per week for NuStats and NIST project staff. 

Face-to-face interviewing began in mid-November. Each surveyor was monitored in a "live" interview 
prior to being allowed to operate independently. A group debrief of surveyors was held in late November 
to trouble-shoot any challenges that had arisen in the first few interviews and also to ensure that all 
surveyors understood the interviewing requirements and were successfully carrying these out. The face- 
to-face interviews began slowly due to lack of sample. Most of the referrals for these interviews were to 
come from the CATI survey and that did not start until early December 2003. During this interim period, 
interviews were conducted with the trickle of respondents who entered the frame through the Hotline. 
Interviewing continued through the end of April to maximize the ability to locate and find potential face- 
to-face interview respondents. 



NIST NCSTAR 1-7B, WTC Investigation 1 1 



Chapter 2 

In all, NuStats had access to 744 potential face-to-face interview respondents, the majority of these 
coming from the screening of respondents in the CATI survey. The face-to-face sample pool included: 

• 456 cases from the CATI, 10 

• 140 cases from the published accounts database, 11 

• 106 cases from the Port Authority's badge list for WTC 7, and 

• 42 referrals from a variety of sources including NIST. 

All of these cases, other than the cases from the CATI, required intensive tracking using LexisNexis to 
locate contact information. Approximately 70 percent of persons in the sample pool were located and 
contacted. One of the anticipated challenges in this interviewing effort was the "no-show" rate for 
interview appointments, given the sensitive nature of the interviews. During December and January, 
NuStats experienced a no-show rate of 1 in 4 (25 percent). However, starting in February the no-show 
rate continued to increase to more than 1 in 3 (36 percent). This was attributed to a shrinking pool of 
available participants; most of them being persons who previously cancelled or were no-shows. As 
people were successfully interviewed, the more reluctant subjects were the ones who remained to be 
interviewed; thus, increasing the likelihood of a "no-show" appointment. In March and April, 
interviewers were trying to interview the "hardest-to-get" cases. 

Interviewing was done at various locations, depending on the preference of the respondent. The goal was 
to maximize participation in the face-to-face interviews by making the interviewing situation as 
comfortable and convenient as possible for the potential respondents. Interviewing locations included the 
Diversity Services offices in midtown Manhattan, NuStats' procured space in downtown Manhattan, as 
well as the work or residential locations of respondents. About 20 percent of interviews were conducted 
at off-site locations. Surveyors were given personal Pen Tablet PC's to accommodate interviewing at 
these multiple locations. The average length of the face-to-face interviews was 3 hours. 

2.2.4 Face-to-Face Interviews with Family Members 

The conduct of the face-to-face interviews with family members was particularly delicate to implement 
because of the emotional duress that many of the individuals still felt. The names and contact information 
(for some not all) of 16 potential respondents were provided to NuStats by NIST. NuStats mailed pre- 
notification letters to those cases where the LexisNexis search provided address of known names of 
family members of victims. These cases were followed up by a telephone call to the person. In all but 
one case, the 8 completed interviews with the family members of victims were conducted by telephone by 
Delia Santos, NuStats. One interview was conducted in-person by Dr. Johanna Zmud, NuStats. 



These were persons who met the screening criteria and agreed to be interviewed again using the face-to-face protocol. 
The published accounts database was developed by Dr. Rita Fahy and Dr. Guylene Proulx, and published as NIST 
NCSTAR1-7A. 
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2.3 FOCUS GROUP INTERVIEWS 

The goal of the focus groups was to elicit group representations of specific events or themes. Distinct 
categories of persons were provided by NIST based on preliminary analyses of CATI survey and face-to- 
face interview data. These groups were prioritized based on expectations of sample availability and 
information value as presented below. The goal was to recruit five to six persons for each group. 

Priority Group 1 

1. Mobility Challenged - WTC 1 only. At least half of the participants were to be wheelchair bound 
or use some type of device to aid their mobility. Half could have less visible mobility challenges, 
such as asthma. Questioning priorities were to (1) have them tell their stories, (2) talk about the 
building design or evacuation procedure policies that aided their evacuation or (3) hindered their 
evacuation, and (4) what would they like to see in terms of building design or evacuation 
procedures policies in the future. These persons would be recruited from the face-to-face 
interview sample. 

2. Persons near the floor of impact in WTC 1 (floors 88 - 94). This group provided explanatory 
power to the causal models on evacuation behavior. Dependent variables in the models are: 

(1) length of time between their decision to evacuate and their actual evacuation time, and 

(2) time out of the building once they started their evacuation. NIST provided variables that were 
identified as independent variables in the model so that these could be probed during the group 
discussion. These persons would be recruited from the face-to-face interview sample. 

3. WTC 1 respondents who started evacuating 2 standard deviations after the mean evacuation time. 
The "glue" of this group was to examine why some people took so long to start evacuating. 
These persons would be recruited from the CATI sample. 

Priority Group 2 

1. 12 Persons from floor 78 (above impact) in WTC 2 who survived the impact. It would be 
important to interview these persons as face-to-face interviews first to know what they did to 
survive. A Lexis Nexus search was performed in order to locate these individuals. 

2. Group who stayed behind in WTC 1, floor 68 for 20-45 minutes after impact. Information 
priority was to find out why they stayed behind and did not start evacuating right away. These 
persons would be recruited from CATI or face-to-face completed interviews. 

3. Floor wardens in either WTC 1 or WTC 2. These were persons who "tried" to perform their 
warden duties. Information priorities were to probe on why people might have ignored them, 
what they felt might have been a better evacuation procedures than the one that they employed, if 
they thought the evacuation procedures / policies they followed were sound and should be 
guidance for future events. These persons would be recruited from CATI or face-to-face 
completed interviews. 

Priority Group 3 

1. People redirected in stairs in WTC 1. These were persons who were evacuating down the stairs 
and were redirected to another stairwell. The information value would be in knowing who did the 
redirecting and why. These persons would be recruited from face-to-face interview sample. 

2. People who were evacuating down the stairwells and were stopped on the sky lobby on floor 78 
in WTC 2 before the plane hit and told to return to their floors. Questions would focus on who 
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did the redirecting; why; and people's reactions to being redirected. These persons would be 
recruited from the face-to-face interviews. 

3. Building support staff. These were elevator operators, maintenance persons, etc., to get their 
assessment of the building design features that aided or hindered successful evacuation. They 
would be recruited through snowball sampling, outside of the CATI or face-to-face interview 
sample. 

4. Evacuation helpers. These were persons in either WTC 1 or 2 who helped an injured person, a 
mobility challenged person, or anyone else during their evacuation to get the helper perspective 
on what building design or evacuation procedure policies helped or hindered their efforts. They 
would be recruited from face-to-face interview sample. 

5. Persons who observed the fireball in lobby of WTC 1. This was a low priority because of the 
lack of sample to constitute a group. 

2.3.1 Focus Group Recruitment and Interview Protocol 

After prioritizing and analyzing the availability of sample with which to recruit focus group participants, 
it was decided to conduct six focus groups: (1) WTC 1 occupants on March 9, 2004, (2) WTC 2 
occupants on April 1, 2004, (3) mobility challenged persons on April 3, 2004, (4) persons with building 
responsibility on April 27, 2004, (5) persons near the floors of impact on April 28, 2004, and (6) persons 
with floor warden responsibilities on May 12, 2004. Even though these groups were vetted for inclusion 
based on the availability of sample, it was still a challenge to locate persons who were willing and able to 
attend. The original plan was to conduct all the groups during one week in New York City. But due to 
the challenge in recruiting participants, the groups were spaced out as noted above. In most cases, 
participants had already completed either a CATI interview or a face-to-face interview. Recruitment of 
focus group participants was conducted by the face-to-face interviewers; whenever possible, the 
recruitment contact was made by the surveyor who had conducted the face-to-face interview with the 
respondent. 

The interview protocols for the focus groups followed the same basic structure, but each was customized 
for specific information priorities. Dr. Carlos Arce, NuStats, moderated the first group. The remaining 
five groups were moderated by Dr. Johanna Zmud, NuStats. Because audio taping or videotaping was not 
allowed, two note takers were present at each group to record the discussion. Each focus group lasted 
approximately three hours, and participants were reimbursed for their travel with a stipend of $100. 



14 NIST NCSTAR 1-7B, WTC Investigation 



Chapter 3 

Summary of Data Collection Outcomes 



Interviews were completed with surveyors of the WTC attacks via CATI, CAPI, and focus groups. The 
information resulting from these interviews provided unique first-hand accounts of experiences in the 
towers on the morning of September 11, 2001. The level of effort required to complete interviews with 
survivors of the WTC attacks was greater than anticipated. There were several factors that influenced 
NuStats' ability to locate, contact, and complete interviews, including availability of sample and the 
willingness of survivors to revisit their experiences in an interview format. The sections below document 
outcomes of the interviewing efforts and the factors that influenced these outcomes. 

3.1 TELEPHONE INTERVIEWS 

A total sample of 26,000 was drawn, comprising 13,000 names for each tower. From this sample, CATI 
interviews were completed with 803 persons. Of which, 427 persons were in WTC 1 on September 11, 
and 376 persons were in WTC 2. Using guidelines of the Council of American Survey Research 
Organizations (CASRO), the estimated response rate was 23 percent. This response rate was driven by an 
inability to contact and screen eligible respondents, rather than by refusals. Of those contacted, only 
7 percent refused to be interviewed. On the other hand, only 47 percent of sampled persons were 
contacted, of which only 1 8 percent were eligible to be interviewed (that is, they were in the towers on the 
morning of September 11, 2001). 

Table 3-1 summarizes the final disposition of the CATI sample and the total (locating) sample. The table 
is comprised of two sets of rows. The top set pertains to the CATI sample and represents those sample 
persons for whom an initial telephone number was identified prior to commencing the CATI survey 
operations. The bottom set of rows with the heading "Total Sample Disposition" represents the results of 
locating/tracking efforts used to identify usable telephone numbers associated with the sample subjects. 
(Recall that only name, SSN, and employer were available; no other contact information was readily 
available.) 

The top set of rows in Table 3-1 presents the final disposition of the sample by tower as well as for the 
overall sample. Several statistics in the percentage distribution (rightmost) column are notable. First, 
subjects for half the sample (50.5 percent) could not be contacted, due either to failures to answer the 
phone, answering machines, unusable numbers (e.g., wrong number, disconnected, business), etc. Most 
of these telephone numbers represent "unloadable" subjects - subjects for whom the initial telephone 
number was incorrect. It bears reiterating that substantial additional research during CATI operations was 
conducted using powerful subscription-based web-based search engines. Unfortunately, little information 
was available for these individuals. 
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Table 3-1. Disposition of the CATI samp 


e and the total sample by tower. 


CATI Disposition: 


WTC l a 


WTC 2 a 


Total 


% Dints 


Interview 


427 


376 


803 


4.0 % 


Partial Interview 


47 


37 


84 


0.4 % 


9/11 decedent 


20 


40 


60 


0.3 % 


Other decedent 


49 


39 


88 


0.4 % 


Not Eligible 


3,712 


3,752 


7,464 


37.5 % 


Language Barrier 


135 


129 


264 


1.3 % 


Eligible Refused to Interview 


138 


139 


277 


1.4 % 


Other Refusal 


224 


181 


405 


2.0 % 


Respondent not Interviewed 


247 


168 


415 


2.1% 


Can't contact/locate Respondent 


4,987 


5,076 


10,063 


50.5 % 


CATI TOTAL 


9,986 


9,937 


19,923 


100.0 % 


Total Sample Disposition 


WTC1 


WTC 2 


Total 


% Dints 


Found initial telephone number 


9,986 


9,937 


19,923 


76.6 % 


Unable to find a telephone 
number 


3,014 


3,063 


6,077 


23.4 % 


SAMPLE TOTAL 


13,000 


13,000 


26,000 


100 % 



a. Table data is weighted. Tower location as indicated in the badge list and may differ from reported 
tower location. 

The bottom set of rows shows that telephone numbers were identified for just over three quarters 
(76.6 percent) of the sampled subjects. Moreover, this rate was fairly uniform across towers. The 
19,923 individuals with an initial telephone number were then loaded into the CATI sample management 
system for calling. Ultimately, all reserve samples were used in the telephone survey. In the initial 
design parameters, it was assumed that 82 percent of the subjects would be locatable. While 76.7 percent 
is close, many of the numbers were obsolete (e.g., disconnect, wrong number) and necessitated additional 
tracking during CATI operations. Ultimately, by the end of data collection, only half the sample 
represented confirmed contacts with subjects. 

A second result of interest is the prevalence of ineligible subjects — those not in the building on the 
morning of September 11, 2001. A third result is the existence of decedents — some from the 
September 1 1 attack and others from causes not necessarily related to September 1 1 (e.g., cause 
unknown, natural causes). Most of the September 1 1 decedents were encountered due to a difference in 
the full (formal) name of the subject and the name that appeared on the badge list (e.g., the badge list 
sometimes contained maiden names, middle names, nicknames, misspelled first or last names, out of 
sequence names, titles, and so on). This impeded the ability to remove known decedents prior to calling. 

The outcome of CATI operations on the final outcome rates is presented by tower in Table 3-2. The table 
shows screening rates, interview rates, and rates of eligible occupants (among those who responded to the 
screening questions). The first row shows that screening response rates were relatively uniform across 
towers at about 46 percent. A screening response rate of 65 percent had been planned. Similarly, 
interview response rates (among screened eligible subjects) were relatively stable across towers at about 
49 percent. This is consistent with the planned interview response rate of 50 percent. 
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The eligibility rates were higher than expected — about 18 percent overall compared to the 14 percent 
expected. The eligibility rate among WTC 1 subjects was slightly higher than those of WTC 2. 
However, the overall response rates are essentially uniform across towers, at 22.6 percent. 



Table 3-2. Su 


mmary disposition rates 


3y tower. 


Disposition Rate" 


WTC 1 


WTC 2 


Total 


Screen 


46.5 % 


45.8 % 


46.1 % 


Interview 


48.6 % 


49.5 % 


49.0 % 


Eligibility 


18.9 % 


16.7 % 


17.8 % 


Overall 


22.6 % 


22.7 % 


22.6 % 



a. Definitions for "Rates" consistent with American Association of Public 
Opinion Research (AAPOR) Standards, which may be found at 
http://www.aapor.org/pdfs/standarddefs2004.pdf 

3.2 FACE-TO-FACE INTERVIEWS 

The face-to-face interviews were conducted via non-probability methods. Thus, response rate estimates 
cannot be derived. In total, 220 interviews were completed between mid-November and mid- April. The 
following is the breakdown of completes by group affiliation: 12 



28 = Near floors of impact 

2 = Persons in the lobby who saw fireballs 

8 = Family of victim 

7 = Persons in building 7 

33 = Persons with building responsibility 

15 = Persons in elevators 

13 = Persons with disabilities 

114 = Persons who evacuated from WTC 1 or 2 not otherwise coded. 



Most of the interviews were conducted with persons who had also agreed to complete a CATI interview. 
The provision of sample, outside of the CATI interviewees, was less than originally anticipated. 
Snowball sampling proved less than effective as survivors were reluctant to provide names of other 
persons who may have qualified for a face-to-face interview out of respect for the persons' privacy or 
mental health. The biggest challenge during the face-to-face interviews was the "no show" rate for 
scheduled appointments, which ranged from 25 percent at the start of interviewing to 36 percent as 
interviewing ended. Interviewers averaged 1 interviews per week in December and January. But this 
productivity dwindled to less than three per week in February, March, and April as available sample and 



Some individuals may have had more than one group affiliation, but each individual was coded to a primary group for tracking 
purposes. A hierarchy was used to do the primary coding based wherein an individual was coded to the hardest-to-locate 
group. 
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willing respondents diminished. Interviewers also found that they were required to travel to residences or 
work places outside of Manhattan for these harder-to- interview persons. 

3.3 FOCUS GROUP INTERVIEWS 

Six focus groups were completed. Planning and recruiting for all focus group participants began in mid- 
March. Recruiting for some focus groups was more challenging than others. The easiest to recruit groups 
were with survivors from WTC 1 and WTC 2 on their evacuation experiences. These groups were held 
on March 9, 2004 and April 1, 2004. Six individuals were recruited for the group of WTC 1 survivors, 
and all six showed at the facility. Eight persons were recruited for the group of WTC 2 survivors and 
seven of the eight showed. 

For the remaining focus groups, it was challenging to track, locate, and recruit individuals. The focus 
group with mobility challenged individuals was held on April 3, 2004. Five individuals were recruited 
(of which two were wheelchair-bound), and three individuals showed up, of which one was wheelchair- 
bound. Focus groups with persons having building responsibility and with persons near the floors of 
impact were held on consecutive days, April 27 and 28, 2004. Eight individuals were recruited for the 
focus group of persons with building responsibility, and six individuals participated. Six persons were 
recruited for the group of persons near the floors of impact, and four participated. Finally, the floor 
wardens focus group was the most difficult to recruit. Only four persons were recruited, and two 
showed up. 

The venues for the focus groups were changed from Manhattan to outside of Manhattan depending on the 
preferences of focus group respondents. Among respondents who were willing to participate in the focus 
group, several no longer traveled into Manhattan. Of the persons who participated in the groups, all were 
active in the discussion and willing to share their experiences, thoughts, and recommendations. 
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NuStats was responsible for delivery of databases containing information resulting from the interviews 
with survivors. These databases comprised various formats, specific to the type of interview conducted. 
In addition to delivering databases, NuStats also provided database support services. These services 
included locating a subcontractor to provide qualitative analysis support for the face-to-face interviews 
and providing statistical expertise to support model development using the CATI data. 

4.1 TELEPHONE SURVEY INTERVIEWS 

The CATI data was delivered in SPSS and DBF formats in late January 2004, after completing processing 
and quality review. After delivery of the final data, NuStats participated in a weeklong analysis session 
with NIST to finalize the statistical analysis procedures. During this time and subsequently, NuStats 
provided data support to NIST for reporting writing and final data review. NuStats also provided 
statistical analysis support during the months of February and March for the development of a dataset that 
was consistent with the causal modeling requirements and the running of necessary statistical programs to 
support quantitative analysis and causal modeling. 

4.2 FACE-TO-FACE INTERVIEWS 

NIST selected ATLAS. ti as the software system for qualitative analysis. NuStats provided assistance to 
NIST in locating and managing a qualitative research expert to support its qualitative analyses. 
Subcontractors for this effort were the co-directors of the survey laboratory of the University of Chicago 
for the National Opinion Research Center (NORC), Dr. Martha van Haitsma and Dr. Virginia Bartot. 
These two individuals provide support to NIST in developing the data analysis plan and coding scheme, 
as well as in ATLAS. ti software usage. After interview data cleaning and editing, which was done in 
Excel, the face-to-face interview data was converted to conform to the specifications of the ATLAS.ti 
software. NuStats prepared and delivered face-to-face interview data on a continuous basis throughout 
the data collection period. 

4.3 FOCUS GROUP INTERVIEWS 

Focus group interview summaries were delivered to NIST as documents in Microsoft Word. Data 
confidentiality constraints prevented audiotaping or videotaping of the focus group sessions. Thus, 
summaries were prepared by the focus group facilitator from detailed notes provided by observers at the 
groups. 
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This document provides documentation of the NuStats work plan, following a kick-off meeting with 
NIST staff and three outside experts on June 9-10, 2003. This work plan updates both the NIST-produced 
white paper, "Collection Methodology for Work Trade Center Evacuation and Emergency Response: 
Telephone Interviews, Face-to-Face Interviews, Focus Groups and Population Sampling" (included as 
part of the original contract solicitation), and the Scope of Work (task-by-task description) that was 
included in NuStats proposal. 

A.l BACKGROUND 

The goal of the National Institute of Standards and Technology's World Trade Center Investigation is to 
investigate the building construction, the materials used, and the technical conditions that contributed to 
the outcome of the World Trade Center (WTC) disaster. The results of the Investigation will serve as the 
basis for improvements in the way buildings are designed, constructed, maintained, and used; improved 
tools, guidance for industry and safety officials; revisions to codes, standards, and practices; and 
improved public safety. The primary objectives of the NIST-led technical investigation of the WTC 
disaster are to: 

1. Determine why and how WTC 1 and 2 collapsed following the initial impacts of the aircraft 
and why and how WTC 7 collapsed; 

2. Determine why the injuries and fatalities were so high or low depending on location, 
including all technical aspects of fire protection, occupant behavior, evacuation, and 
emergency response; 

3. Determine what procedures and practices were used in the design, construction, operation, 
and maintenance of WTC 1,2, and 7; and 

4. Identify, as specifically as possible, areas in building and fire codes, standards, and practices 
that are still in use and warrant revision. 

The NIST Investigation Plan can be found at http://wtc.nist.gov, including a description of Projects 7 
and 8. Under Project 7, Occupant Behavior, Egress, and Emergency Communications, first-hand 
accounts of the events of September 11, 2001, from inside WTC 1, 2, and 7 will be collected. This data 
collection effort will evaluate the role of occupant behavior and evacuation technologies and practices for 
tall buildings, including decision-making and situation awareness, time-constrained evacuation strategies, 
communications, role of floor wardens and fire safety directors, and issues concerning people with 
disabilities. Additionally, NIST will seek specific observations of fire and smoke conditions and/or 
structural damage from within the building. Families of the victims, who communicated with loved ones 
inside the towers before collapse, will be interviewed to determine the nature of the environment above 
the floors of impact. The objectives of Project 8, Fire Service Technology and Guidelines, are to build 
upon work already done by the Fire Department of New York (FDNY) and McKinsey & Company by: 
(1) fully documenting what happened during the response by the fire services to the attacks on the 
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World Trade Center, up to the time of collapse of WTC 7; (2) identifying issues that need to be addressed 
in changes to practice, standards and codes; (3) identifying alternative practices and/or technologies that 
may address these issues; and (4) identifying R&D needs that advance the safety of the fire service in 
responding to massive fires in tall buildings. Thus, a subset of the emergency responders who were 
present at the WTC complex will be asked to voluntarily participate in the face-to-face interview or focus 
group phases of this project. Only first responders who participated in fire suppression, operational, or 
search and rescue activities prior to the building collapse will be considered for inclusion in these phases 
of the study. 

A.2 OVERVIEW OF METHODOLOGICAL APPROACH 

The WTC interviews are being conducted to assist the NIST-investigation team in discovering "what 
happened" from the time of the first airplane hit until the collapse of the buildings (WTC 1 , WTC 2, and 
WTC 7). The interviews are focused on behaviors and observations that took place within the buildings 
and up to a block radius surrounding the buildings. The interviews are not concerned with events 
subsequent to the collapse of the buildings or outside of this spatial radius. The data collection objective 
is to conduct 800 telephone interviews with occupants of WTC 1 and WTC 2; up to 600 face-to-face 
interviews with occupants of WTC 1, WTC 2, and WTC 7, 30 face-to-face interviews with families of 
victims, and 150 face-to-face interviews of first responders; and 10 focus group sessions with first 
responders and five (5) focus group sessions with building occupants. The types of information to be 
derived from these data collection activities include: 



• 



• 



Identifying physical and human impedances and facilitators to building evacuation, 
Developing a better fundamental understanding of: 

o Egress 

Human behavior 

Emergency communications 

o Emergency response procedures 

Building/ response technology. 

This multi-methodological approach was selected to uncover the required information for several reasons. 
First, multiple methodologies increase confidence in the conclusions and findings when more than one 
methodology arrives at the same conclusions. Second, the varied objectives of the Investigation mandate 
complementary approaches to accomplish all the goals. Finally, concerns associated with the time latency 
since September 1 1, 2001, suggested the use of different approaches and techniques in order to increase 
memory recall and accuracy. 
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A.3 TASK 1: INSTRUMENT DEVELOPMENT, PROTOCOLS AND DATABASE 

DESIGN 

A.3.1 Telephone Interview: Survey Instrument 

One of the data collection methodologies is the telephone interview. The collection mechanism utilized 
in this phase of the study will be a computer-assisted telephone interview. It is expected interview will 
range from 20 - 30 minutes, with an average length of 25 minutes. The primary goal of the telephone 
interview is to provide statistical estimates that will inform both the NIST Investigation and evacuation 
theory, particularly as it pertains to disasters involving fires. The categories of information to be captured 
in the telephone interviews include: 



Means of first alert (i.e., emergency notification) 

Situational awareness 

Factors associated with evacuation decision 

Delay period and activities associated with delay in evacuation 

Evacuation route choice and reason for choice 

Positive and negative aspects of egress 

Pre-existing injuries or injuries incurred during evacuation 

Occupant characteristics and traits 

Employer-related information relevant to the investigation, such as existence or provision of 
emergency training 

Physical building elements and human-related impedances or facilitators encountered during 
evacuation. 



NuStats will use information obtained during the kick-off meeting to draft the telephone interview 
instrument. Such information includes verbal communications with NIST staff and the outside experts, 
the documents identifying key questions of the Investigation by project, the document developed by E. 
Kuligowski on Sample WTC Questions, the notes from the brainstorming session of the WTC project 
leaders and team members pertaining to survey content, and questionnaires provided by G. Proulx for 
prior surveys on this topic. The questions will flow in a logical order in relation to the chronology of the 
events, as suggested in the literature. 

NuStats will follow standard telephone interview construction techniques. The first draft of the telephone 
interview instrument will be reviewed in a joint meeting with NIST staff and the outside experts and 
subsequently revised. Prior to this joint meeting, our consultants, Dr. Jon Krosnick and Jamie Abelson, 
will review the questionnaire for content, flow, and question construction and validity. Once the final 
draft instrument has been developed, NuStats (Heather Contrino) will assist NIST in submitting the 
instrument package for OMB approval. 
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A.3.1.1 Telephone Interview: Survey Population 

The persons who were in WTC 1 and WTC 2 immediately prior to the first aircraft impact on 
September 11, 2001, will constitute the population to be sampled in this study segment. The sampling 
plan of the September 11, 2001, occupants is a stratified probability sample of building occupants. 

Stratifying the population. The proposed stratification criteria are based on the assumption that 
occupants experienced unique egress behavior, observed different aspects of the structural integrity of the 
buildings and suppression systems, and had varied interactions with first responders based on their 
location in the buildings at the time of the airline impacts. The capture of these unique circumstances, 
observations and experiences are critical for NIST in understanding the egress process and the structural 
integrity of the buildings, its suppression protocols, and first responder impact leading up to the time of 
collapse. 

A primary stratification criterion will be building of occupancy - namely, WTC 1 and WTC 2. A 
secondary stratification criterion will be location within the towers as defined by three zones that reflect 
distance from ground floor and proximity to point of impact. The zones are defined according to the 
location of the mechanical floors. Separate zones approximately comprise thirds of the buildings: 

• the top floors (floors 77 to 91 in WTC 1 and floors 77 to 1 10 in WTC 2), 

• the middle floors (floors 43 to 74 in WTC 1 and WTC 2), and 

• the lower floors (floors 9 to 40 in WTC 1 and WTC 2). 

A third stratification criterion is tenant size. The tenant size criterion represents a floor as one of two 
levels: 

• large tenant floor (a single tenant occupies greater than 40 percent of the usable square 
footage of a floor), and 

• small tenant floor (all other tenant-occupied floors). 

Enumerating the population. At this time, NuStats expects to use the security badge list as the sampling 
frame. The list would be sorted to approximate the stratification criteria noted above. The utility of this 
list is pending further investigation of the completeness and reliability of the contact information. If this 
list is used, it will constitute the sampling frame of WTC 1 and WTC 2 occupants. 

This list is highly attractive because of its full coverage of the September 1 1 WTC occupants. The 
downside of the list is that it also contains a large number of occupants who were not in the WTC on the 
day of the tragedy. The list is unable to distinguish who was present in the WTCs on September 11. This 
means that considerable, unanticipated screening will be needed to identify September 1 1 occupants. 

Ideally, security badge list will contain home address and telephone number as of September 11. If it 
does not, then considerable unanticipated resources will be needed to locate the subjects. Even with the 
September 1 1 addresses and telephone numbers, we anticipate that a high proportion of subjects will have 
moved or changed telephone numbers and this will require an unanticipated incremental locating effort. 
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Screening. It is important to note that telephone screening will be needed to determine whether an 
individual on the list is eligible - that is, whether he/she was an occupant of WTC 1 or 2 at the time of the 
first aircraft impact on September 11, 2001. It is estimated that a total of 10,000 to 14,000 people were 
inside WTC 1 and WTC 2 at the time of the first impact. 

Selecting the sample. Stratified probability sampling of individuals will be conducted from the frame to 
yield a total of 800 telephone interviews. Oversampling of certain subgroups is expected, but the 
delineation of those groups has not yet been finalized. Oversampled groups might include: 

• occupants near the point of impact 

• occupants in each tower (e.g., equal allocation to WTC 1 and WTC 2) 

• others to be specified. 

It is assumed that there will be an approximate 30 percent participation rate among those asked to 
participate. A final data set of 800 cases will ensure a 0.05 level of significance and power of 0.80. 
NuStats will make every effort to increase participation above 30 percent. 

A.3.1.2 Telephone Interview: Pilot Testing 

For the CATI pilot, NuStats will randomly select 10 building occupants from the list provided and 
attempt to interview them. Our subcontractor, DataSource, will conduct these interviews. The Task 
Manager, Heather Contrino, will be on-hand to directly interact with interviewers during the pilot phase. 
DataSource has technology for remote monitoring of interviews so that NIST staff can assess the 
interview administration, if requested. All aspects of the data collection plan will be tested including: 

• Instrument flow; 

• Respondent comprehension; 

• Interviewer administration; 

• Identification of questions needing probes or clarifications; 

• Data coding and processing; and 

• Review of survey data. 

A.3.2 Face-to-Face Interviews: Survey Instrument 

The objectives of the face-to-face interview segment is to gather richness of detail about the human 
evacuation experience, to obtain first-hand accounts and observations of the activities and events inside 
the buildings on the morning of September 1 1 for specialized investigatory topics, and to capture the 
views, experiences, and first-hand accounts from sub-populations of high interest to the Investigation. It 
is estimated that the average face-to-face interview will last approximately two hours, with some lasting 
significantly longer. 
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The face-to-face instrument design is guided by the proposed methodology, which is a synthesis of the 
Behavioral Sequence Interview Technique (BSIT) originally developed by Keating and Loftus and the 
Cognitive Interviewing Method (CIM), originally developed by Fisher and Geiselman. These two 
interviewing methodologies were developed with the purpose of assisting persons in retrieving more 
comprehensive and accurate memories of incidents, and sharing important attributes. Both approaches 
begin by allowing the informant to retell an unimpeded account without interruption from the interviewer, 
and both initially employ a chronological retelling of information. However, BSIT was designed to yield 
a database of qualitative information that could be subjected to systematic analysis and consolidation, 
while CIM was designed to facilitate investigative interviews. Since the Investigation is pursuing both 
goals (i.e., creation of a database of evacuation-related behaviors and an investigatory attempt to capture 
information relevant to outcomes), the proposed methodology combines these two approaches. 

Cognitive interviewing has been the subject of many empirical investigations. Fisher, et al summarized 
these findings, demonstrating that the methodology significantly increases the amount of information 
recalled without affecting rate of errors. Interviewing a large number of informants will allow 
corroboration of information, thereby compensating for the likely increase in the absolute number of 
errors. Accordingly, it is likely that this approach will be productive in achieving a holistic view of the 
building evacuations. 

NuStats will design three paths within the face-to-face questionnaire: one for occupants, one for family 
members of the victims who communicated with loved ones inside the towers before collapse and one for 
first responders. Depending on initial screening questions, the respondent will proceed through a 
specialized version of the questionnaire directed at their survey population. This will increase design 
efficiency and allow all responses to be stored in one database. 

The occupant questionnaire will be used primarily to capture the egress process, including occupant 
behavior and emergency communications in rich detail. The face-to-face questionnaire for family 
members will collect data that describe, define and measure the nature, extent and timing of 
communications between family members of the victims and their loved ones inside the towers before 
collapse. The first responder questionnaire will collect data on emergency communications, emergency 
response procedures, and building / response technology. As with the CATI instrument, Dr. Jon Krosnick 
and Jamie Abelson will review the questions to ensure they will meet the research objectives and, at the 
same time, minimize unit and item non-response. 

A.3.2.1 Face-to-Face Interviews: Computer-Assisted Personal Interview (CAPI) Format 

NuStats will automate the interview structure to increase efficiency and reduce error. The specific CAPI 
technology recommended is the use of Tablet PCs. By using this state-of-the art technology, high quality 
survey data can be electronically coded and organized, and stored directly in an electronic format, with all 
relevant error checking performed automatically. This process will simultaneously eliminate unnecessary 
data entry costs and associated key entry errors that typically occur after a traditional paper-based field 
survey is complete. In fact, through the use of a Tablet PC for survey administration, the entire survey 
can be captured and coded in real time making data delivery faster and easier. 

Tablet PCs offer laptop PC functionality and processing power - in a form factor that is significantly 
smaller than a laptop. An example of a Tablet PC can be seen in Figure A-l. 
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Figure A-l. Example pen tablet PC. 

This pen tablet PC weighs only 3.1 pounds, has a 10.4-inch viewable screen, and comes standard with an 
800 MHz CPU, 128 MB of RAM, and a 20 GB hard drive. In addition to the standard touch screen 
display, the latest generation of Tablet PCs includes a wireless keyboard to facilitate data entry. The 
Tablet PC shown in Figure 1 converts easily from a Tablet PC to a Notebook PC form factor. Our 
proposed instrument design using this technology is described in the following paragraphs. The 
instrument will be developed (i.e., programmed) to collect the fact-to-face interview data in three stages, 
following the steps of the combined BSIT / CIM interview process. 

Step 1: Unimpeded open-ended narrative account. Both BSIT and CIM begin the process by asking 
the participant to chronologically recount his or her story. The proposed starting point is when it became 
apparent that something unusual had occurred on the morning of September 11, 2001. The proposed 
ending point is when the participant feels that he or she reached a location where they felt safe (or, 
alternatively, when he or she successfully reached the exterior of the building). Researchers and 
practitioners involved with cognitive interviewing believe that starting the face-to-face interviews in this 
manner both improves recall and helps build rapport between the participant and the interviewer. 

NuStats will expand on the guidelines for step 1 by organizing the interview process using the sensory 
approach. Respondents will be asked something like: I'd like you to share your evacuation experience 
with me from the time you first became aware that there was something unusual occurring until you were 
safety out of the building. I'm particularly interested in specific things you saw, heard, smelled, or 
touched during your evacuation. 

The instrument for this phase is a list of sensory experiences that the respondent may mention in their 
narrative. The interviewer only "taps" the screen to record that a particular type of sensory experience 
was mentioned. For example, the participant might briefly mention an odd odor which the interviewer 
makes note of. During the negotiated phase (step 2 below) the interviewer will want to prove this fact to 
determine whether the smell might have been that of jet fuel, smoke, or of some other origin as yet 
unknown. 
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Step 2: Structured (negotiated) narrative account. After participants complete their stories, 
interviewers will prompt them to go through the story again, but this time they will work cooperatively 
with the interviewer to record entries into a table. This approach is employed by BSIT for three primary 
reasons: (1) to yield a structured account that can be entered into a database without further processing; 
(2) to avoid the biasing effects of having interviewers ask specific questions; and, (3) to enhance the effort 
at recall put forward by participants by encouraging their active collaborative participation, an advantage 
to open-ended formats as noted by Fisher, et al. 

Each row of the table will represent a single action in a sentential format, meaning that each action is 
expressed as a grammatical sentence. The approach is based on the hypothesis that people encode 
episodic memories in a manner consistent with this format, thus facilitating both recall and data entry. 
Each column of the table represents three essential components of actions: a cue, an action, and the reason 
for taking that action (see Table A-l). 

Table A-l. Example tabular face-to-face interview data entry. 



Cue 


Action 


Reason 


I heard but couldn't see someone 
yell "I've found a clear path" 


So I stumbled in the dark towards 
where I thought the voice came 


So that I could find a way to 

escape 


My path was blocked by debris 


So I called out to whoever yelled, 
"I'm near the reception area. 
Where are you? 


To try to get a better idea of 
where the person was 



Cues can be either external (e.g., signs of a fire, someone saying something) or internal (e.g., 
remembering about another means of escape). For purposes of this research, cues will be sensory 
experiences. Actions are expressed using specific action verbs (i.e., ran. instead of went) and may include 
artifacts (e.g., a fire extinguisher) used by the informant. Reasons are the intentional, goal-directed base 
for the action. The interviewer will encourage the participants to use their own words to the greatest 
extent possible. 

Experimental findings in psychological research on memory suggest that when people perform actions, 
their abilities to verbally recall those actions are significantly improved. Script theory suggests that 
people naturally organize their knowledge of actions using narrative sequences of actions structured 
around their pursuit of goals. However, gaps in the narrative are anticipated, especially given the long 
period of time that will have elapsed between the event and the interview. The information entered by the 
interviewer in Step 1 on the sensory experiences recounted in the unstructured narrative will be used to 
"populate" a table prior to the start of the negotiated structured narrative report. This pre-populated table 
will be used to structure the re-telling of the respondent's story. In addition, interviewers will assist the 
participants to fill in these gaps by asking them to recall events in reverse order, an approach used in 
CIM. Interviewers will, however, encourage participants to report only those memories about events or 
incidents that they are confident really occurred to them. 

Step 3: Probing for specific information. After completing the negotiated, structured narrative account, 
interviewers will ask specific open-ended questions (probes) intended to provide specific information of 
particular value to the investigation. While some of this information is likely to be part of the structured 
narrative account, participants may be able to recall other valuable information as well. In this step, 
interviewers will ask questions related to topics that have been identified a priori as being priority topics. 
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This will ensure that all information of significant importance to the Investigation will be examined in the 
face-to-face portion. 

Depending on the population, probes may be used to try to elicit information including, but not limited to: 

Location of the informant at the time of certain marker events (e.g., location in WTC 1 when 
WTC 2 collapsed) 

Fire conditions (e.g., fire and smoke); 

Other cues of interest (e.g., the smell of jet fuel); 

Presence and activities of persons with disabilities; 

Use of elevators by self or others; and, 

Knowledge of any obstacles to their progress while using the stairs. 

Because information about many of these areas of concern requires precise responses, questions for open- 
ended probes will be developed collaboratively between NuStats and NIST. Responses to probes may be 
recorded using standardized formats where feasible. For example, all participants who observed smoke 
may be asked to estimate the smoke density using an encodable scale, such as visibility distance. To help 
standardize scales or indexes used and also the reporting of locational information, graphical prompts will 
be included in the software to allow for better specification. 

A.3.2.2 Face-to-Face Interviews: Survey Population 

The populations to be sampled in this study segments include the following types of people: 

• individuals who were in WTC 1, WTC 2, and WTC 7 immediately prior to the first aircraft 
impact on September 11, 2001; 

• individuals who communicated with members of their families who were in WTC 1, WTC 2, 
and WTC 7 immediately prior to the first aircraft impact on September 11, 2001, and who 
never made it out of the buildings safely; 

• individuals who were among the first responders (i.e., firefighters and police) to the WTC 
disaster. 

The sample will be selected on the basis of NIST knowledge of populations that may have unique pieces 
of information to add to the Investigation. These are persons who belong to special subpopulation groups 
or were in locations during the evacuation that enabled them to experience distinctive situations. This 
method of sampling is called purposive or judgmental sampling. The populations of interest for the face- 
to-face interviews are: 

• Occupants near the floors of impact (WTC 1 and WTC 2) 

• Occupants of WTC 7 
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Floor wardens, fire safety directors, and other persons with building responsibility 

People in elevators or lobbies 

Occupants with disabilities 

Family members of victims who called out of the towers 

First responders (members of the Fire Department of New York, Port Authority Police 
Department, New York Police Department, and other having operational responsibility. 

While the exact numbers of interviews for each group has not been finalized, the total number of face-to- 
face interviews will be 750 interviews, of which 150 will be with first responders. 

Enumerating the population. The population will include the entire occupant, management, and first 
responder population of World Trade Center WTC 1,2, and 7. 

Selecting the sample. The sample for the face-to-face interviews is purposive. The potential 
respondents will be located via a number of methods. For example, NIST will provide the names and 
contact information for the first responder sample. NuStats expects to receive approximately 300 names 
from which to select these respondents. Other special populations can be identified through media 
accounts. NuStats will work with Guylene Proulx to identify potential respondents as she is the 
contractor for another WTC-related study of media coverage. A number of the respondents for the face- 
to-face interviews will be identified via the CATI interviews. During CATI interview, it will be noted 
when a respondent has characteristics or experiences matching the subpopulations of interest. In such 
cases, the respondent will be interviewed via CATI and then an appointment will be set up for a face-to- 
face interview. 

A.3.2.3 Face-to-Face Interviews: Pilot Testing 

The pilot of the face-to-face instrument will be done in New York City at MBC Research Center. Johanna 
Zmud (project director), Robert Santos (senior methodologist), and Delia Santos (task manager), will 
conduct the pilot face-to-face interviews. A total of five face-to-face interviews will be conducted among 
occupants, family members and / or first responders. The viewing room at MBC Research will enable 
NIST staff to observe the pilot interviews. As with the CATI interviews, all six (6) aspects of data 
collection will be tested. 

A.3.3 Focus Groups: Interview Protocol 

The goal of the focus group interviews is to elicit accurate group representations of specific events or 
themes. Williams reports that in a group setting, people provide cues that evoke memories in others, and 
that social pressures mediate against reporting misrepresentations of what they recall. The interview 
protocols will build upon the concepts and operationalizations used in the CATI and face-to-face 
instruments. Focus group protocols will be designed to help facilitate recall and elicit group 
representations of specific events, communications, and egress behavior. This will be accomplished 
through a tiered focus group protocol that begins with less sensitive topics and progresses into more 
detailed and difficult information as the session proceeds. 
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A.3.3.1 Focus Groups: Target Population 

Two distinct populations will voluntarily participate in the focus groups: occupants and first responders. 
The first set of focus group interviews will be the occupant sample. Distinct categories of people will be 
selected for inclusion in this study. The objective of this study is to capture the experience of people in 
unique places in WTC 1, 2, and 7. Some of these groups have been identified by NIST: the 16 people 
who were above floor 78 who made it safely out of WTC 2, disabled persons, persons who saw the lobby 
fireball, and persons who were trapped in the elevator. Every effort will be made to include no less than 
5 people in each of these categories in this study, with 10 people constituting the preferred focus group 
size. NuStats anticipates conducting five occupant focus groups. 

First responders will constitute a second set of focus group interviews. The set of first responders will 
include FDNY, NYPD, PAPD, and other groups identified as having operational or command authority at 
the World Trade Center on September 1 1th. The focus group size will be determined as an operating unit 
size, if applicable. An operating unit may be a Fire Department company, for example. This project 
proposes 10 focus groups, each containing five people. 

Sample selection. The people selected for inclusion in the focus groups will be selected using non- 
probability sampling procedures. Respondents contacted for the CATI interviews or face-to-face 
interviews may be eligible for participation in the focus groups. In addition, respondents in these other 
studies will be asked to provide the names and contact information for people they know in each of the 
categories of interest. The contractor will collect names until at least five people in each category have 
agreed to participate in an occupant focus group, with a preference for 10 people. It is expected that the 
potential participant list for the first responder focus groups will be provided by NIST. The same process 
will occur for selection of the first responder samples, with a preference for inclusion of entire operating 
units (about 5 people per unit). 

A.3.4 Instrument Evaluations 

Each instrument will be evaluated prior to pilot testing. The instrument review process will benefit from 
using cognitive interviews to identify, measure, and/or reduce survey response errors. The administration 
of this survey requires respondents to accurately report factual data for which knowing, remembering, and 
placing events in time are critical. Cognitive interviews of likely respondents would serve to identify and 
assess the potential for survey response errors by examining comprehension of the survey item, retrieval 
of relevant information, judgment based on recall, and mapping and reporting of a response. 

Cognitive testing is relatively new to the survey industry (it has come into use only in the past 1 5 years or 
so). It involves the observation and testing of prototype respondents in a controlled laboratory setting to 
provide insight into the cognitive survey process that is undertaken by a subject who is interviewed by an 
interviewer. The cognitive survey response process literally refers to what happens in a subject's mind 
between the time he/she is asked a question, up to and including the point that he/she provides a response. 
It is composed of four stages: (a) comprehension; (b) retrieval of information; (c) response formation; 
and (d) response editing. Cognitive testing provides insight into these areas in a way that can lead to 
significant improvements in question wording and/or validation of research constructs. 

Not all questions in an instrument require cognitive testing. Dr. Jon Krosnick is an expert in the cognitive 
survey process and he will assist in the identification of the questions that do require cognitive testing. 
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Our approach would be to identify in the review process a subset of critical questions and constructs that 
might benefit from cognitive testing (regardless of the mode of administration - CATI, face-to-face, or 
focus groups), and to employ a small number of cognitive interviews with likely respondents to address 
them. Cognitive tests will be done with individuals known to NuStats who were building occupants on 
September 1 1 . 

A.3.5 Finalize Survey Instruments and Interview Protocols 

After completion of the pilot tests, NuStats will provide a report that details recommended changes in the 
questionnaire/ protocols and training materials. This report will include a discussion of recommendations 
for modifying question order / sequencing, introductory statements, response categories and question 
wording that might be problematic for the respondent. The report will also include a discussion of 
recommendations for modifying interviewer instructions. 

A.3.6 Pre-Field Work Meeting 

A working session with NIST and its outside experts will be held in which NuStats details problems, 
identifies questionnaire items that need revision, and suggests corrections for problems encountered 
during the pilot. Following this meeting, NIST will notify NuStats of any recommended changes in the 
survey instrument and protocols. NuStats will incorporate only NIST-approved changes to the 
instruments and in-person protocols before going into the field. 

A.3.7 Protection of Human Subjects and IRB Approval 

This data gathering effort will ensure that all precautions required by the Common Rule for the Protection 
of Human Subjects are met or exceeded by the contractor. Participation in any part of this project by any 
person will be strictly voluntary. Interviewers will be trained to establish a rapport with participants 
based on a compassionate interest in their story and will ensure participants that information provided will 
be of value in preventing casualties in future building emergencies. During the briefing, interviewers will 
provide information to participants about where and how to receive counseling and about the fact that 
participants may stop the interview at any time without explanation. Interviewers will also be trained to 
recognize signs of post-traumatic stress. Similar services will be offered to participants of focus groups 
and to people taking the telephone interview. NuStats will take the necessary precautions to ensure the 
safety of contract employees administering, collecting, or otherwise involved in this data collection effort. 
Finally, Nustats will ensure that the identities of the subjects are held in the strictest confidence. 

The human subjects protocols to be used in this project by NuStats and DataSource will be reviewed by 
Essex Institutional Review Board. 

A.3.8 Training of Surveyors 

NuStats will train a team of survey specialists for CATI and face-to-face administration as well as focus 
group moderators. Training protocols will be prepared by each of the Task Managers (Heather Contrino, 
Delia Santos, Kim Hilsenbeck) under the direction of the Project Director, Dr. Johanna Zmud. All 
interviewers will receive an eight-hour training session and will be required to perform simulated 
interviews before beginning actual data collection activities. NuStats produces project-specific 
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interviewer-training manuals for each project. The basic elements of the manual include an overview of 
the study, its objectives, glossary of terms, and questions and answers typically posted by the respondent. 
Another aspect of training deals with the intricacies of the survey instruments themselves, with separate 
training for each survey instrument involved. 

In this case, training materials and sessions will include specific provisions to address the unique needs of 
this study including: 

• Informed consent; 

• Privacy/confidentiality; 

• Elements of risk; and 

• Provisions for the protection of human subjects. 

Mock interviews are used to help surveyors quickly become familiar with the survey and nuances of the 
formatting and layout. The mock interviews will also serve to help face-to-face interviewers practice 
handling encounters with different types of respondents. Surveyors will not begin working on this study 
until they have passed a project-specific test. 

The telephone interviewers will be regular staff of DataSource. Interviewers for the face-to-face 
interviews will be contracted social work practitioners. Even though the social workers will be sensitive 
to context in which the interviews are being done, all interviewers and moderators will receive specialized 
training. Our consultant, Jamie Abelson, will be particularly active in this training component. This 
component will include techniques for dealing with emotionally traumatized respondents, as well as voice 
tone and emotional mediation techniques. The contracted social workers will also be trained specifically 
in BSIT and CIM techniques using in-house experts and outside practitioners. 

CATI interviewers will be trained at DataSource, NuStats' subcontractor, while face-to-face interviewers 
will be trained at NuStats' space in New York City. Since focus group moderators are all in-house staff, 
Ms. Abelson will brief them at the NuStats' offices in Austin or in the D.C. area if the NIST staff would 
care to attend. 

Both NuStats key staff and DataSource supervisors will continually monitor interviewers and moderators 
involved in the study to ensure the highest level of quality is maintained. Interviewer debriefmgs will be 
held weekly to address any data collection issues that arise and to heighten interviewers' performance. 

It should be noted that NuStats will require all staff working on this project to sign new legally binding 
pledges of confidentiality specific to this project, in addition to the pledges all staff sign as part of their 
employment contract with the firm. Any additional subcontractors or consultants retained for this project 
will be required to do the same. 

A.3.9 Database Design 

NuStats will design a database, i.e., an encoded table of results, based on the developed coding scheme of 
all concepts and their operationalized measures used in the telephone and face-to-face approaches. Thus, 
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the database will consist of two parts: CATI data and face-to-face data. Focus group information will be 
delivered to NIST as court transcripts. NuStats will design the database structures for the telephone and 
face-to-face surveys according to the NIST specifications. Upon final approval of the questionnaires and 
moderation guides for each component of the study, the NuStats team will develop the database structures 
and fully document variable names, structure, and skip patterns in a comprehensive Data Matrix. One 
data matrix will cover each of the two databases. The Data Matrix will include all variable specifications 
including built in quality assurance measures (such as edit, range, and consistency checks) and will serve 
as the final road map for database development. The number of unique variables comprising the data 
bases has not be specified, however, the RFP states that there will be no more than 75 encoding variables. 

To protect the confidentiality of participants, each data record will be assigned a unique identifying 
number. This will ensure that the names of study participants are not associated with responses to 
questions. Moreover, the data file containing the link between name and ID number are stored separately 
from the data files containing question responses. All confidential information will be stored in 
password-protected files by the holders of this information. 

A.3.10 Additional Data Collection 

NuStats understands that the scenario may arise where an individual critical to developing an 
understanding of the events of September 1 1, 2001, may be unavailable or unidentified during the period 
of performance of the Contractor. Thus, NIST will obtain NIST Institutional Review Board (IRB) 
approval to conduct a limited number of face-to-face interviews with individuals deemed by NIST likely 
to contribute significantly to the outcome of the Investigation. The scope, objectives, and procedures 
used in the additional data collection will be similar to the scope, objectives, and procedures used by the 
contractor. Telephone interviews and focus groups will not be conducted in this additional data collection 
effort. It is anticipated that the number of face-to-face interviews conducted by NIST will be less than 10 
percent of the number of face-to-face interviews conducted by the contractor. The contractor will incur 
no duties or obligations related to the additional data collection. 

A.4 TASK 2: SURVEY ADMINISTRATION 

A.4.1 CATI Field Work 

NuStats subcontractor, DataSource, will reach a participation rate of 30 percent to conduct a minimum of 
800 interviews with building occupants. Due to the length and nature of the interview, we will not 
include partial interviews as completes. 

DataSource will use a small, well-trained team of 8-12 interviewers to conduct the CATI interviews. 
Each interviewer will be experienced in dealing with sensitive subjects, well versed in mental health 
protections and confidentiality issues, and will have passed an extensive training exam before being 
allowed to dial on the survey. 

It is expected that data collection will last for eight weeks. This is to allow for the time it will take to 
reach a particular respondent (research, initial contact, follow-up contact/ interview) and to track those not 
easily reached to achieve the desired participation rate. Dialing hours will be 10 a.m. to 9 p.m. (Eastern 
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Time) and calling will vary across days, nights, and weekends to maximize response. NIST will provide 
the sampling frame to the NuStats team in the format requested by NuStats. 

A phone call will be made to the sampled individuals. NuStats will vary the day and time of the phone 
call to increase the likelihood that contact with the individual will be made. If an individual chooses to 
participate, he/ she will be asked to voluntarily disclose a preferred means of follow-up contact. 
Regardless of the preferred means, all contacted individuals will receive a personalized mailing that 
details the purpose and goals of the study, the client, and the types of the questions that will be asked. 
The mailing of the letters will be done prior to the expected day / time of the follow-up contact. 
Individuals will be instructed to phone a toll-free number at DataSource should they require further 
information. 

Heather Contrino, task manager, will monitor progress through well-established communications with the 
DataSource field team. NuStats project managers receive nightly dialing reports (documenting the 
number of completed interviews) and weekly disposition reports (a cumulative disposition file of all 
numbers dialed to date) from DataSource. Ms. Contrino will customize this reporting to suit NIST 
progress reporting needs. 

NuStats is always concerned with non-response, and understands that the response rate for this survey is a 
critical element of the data collection process. We will use time-tested methods for minimizing refusals 
without infringing on the voluntary nature of this survey. Techniques to combat non-response will 
include: 

• Advance notification 

• The use of Lexis/Nexis to locate "missing" respondents selected for the study; and 

• A toll-free respondent hotline and a respondent web page. 

Advance notification has been proven in studies such as this one to minimize refusals. Whenever 
possible, we will send a pre-notice letter describing the survey. The letter will be short, written in simple 
language, will explain the survey relevance, distinguish it from other surveys investigating the WTC 
disaster, and will be personalized to the extent possible. Respondents will be encouraged to contact 
DataSource through the respondent hotline if they have questions. 

In some cases, contact information will be outdated or nonexistent. NuStats anticipates that we may face 
this and other barriers in locating some of the sampled occupants. We propose to use our in-house 
resource Lexis/ Nexis, for tracking subjects selected in the sample but form whom inadequate or no 
contact information is available. 

A member of our trained interviewer team will work at a toll-free respondent hotline during calling hours. 
Since many respondents will have received advance notification, we anticipate that some will have 
questions or concerns about participating. Making staff available to answer respondent concerns is a 
critical element in building trust and establishing validity in the survey design. NuStats will design and 
informational (with Q&A capabilities) web page for respondents who seek quick answers to their 
questions. 
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A.4.2 Face-to-Face Field Work 

NuStats will conduct 750 face-to-face interviews, with occupants, family members, and first responders. 
NuStats will use the synthesized BSIT and CIM technique in conjunction with CAPI technology to create 
a database of evacuation-related behaviors and capture information relevant to outcomes. NuStats will 
recruit the face-to-face interviewers from the available pool of psychiatric social workers in the New 
York metropolitan area. We expect this team will be trained (and experienced) in dealing with sensitive 
topics and additional mental health provisions. Their experience will be supplemented with project- 
specific training for the WTC investigation. This will include training to recognize signs of post- 
traumatic stress. They will also be able to provide sources for additional mental health counseling, should 
a respondent request it. 

NuStats expects to recruit a team of 8-12 interviewers and to conduct approximately 6-8 interviews per 
day (seven days a week), with approximately 2 hours allocated for each interview. This is a larger face- 
to-face interviewer pool than we identified in the RFP to ensure that the data collection in completed 
within the necessary timeframe. This will require more Pen Tablet PCs than we had budgeted. 
Interviews will not be video- or audio-taped to protect the privacy and confidentiality of the respondent. 

NuStats will try to schedule the location for the interviews to be as convenient as possible. NuStats will 
employ a variety of locations to ensure that a convenient location for each face-to-face respondent can be 
achieved. Locations include NIST space, MBC research space, space in lower Manhattan at 181 Mott 
Street, as well as the respondent's home or work place, if so desired. 

Delia Santos, task manager, will manage all fieldwork. DataSource will handle the recruiting and 
scheduling for all face-to-face interviews. Because DataSource interviewers will go through intense, in- 
depth training on the WTC project, we believe they will be proficient in dealing with this sensitive topic, 
and will be able to best answer questions about study objectives and how results will be used. 

A confirmation letter will be mailed to each individual who agrees to participate in the study. The letter 
will explain the purpose of the interview, describe how the interview will be conducted, confirm the time 
and date of the interview, and provide directions to the interview location. Reminder calls will be made 
to these persons by the actual face-to-face interviewer to whom the case has been assigned. This will help 
establish a relationship between the interviewer and respondent prior to the actual interview. 

Through the use of the CAPI technology, data will be entered and coded almost in real-time. This will 
enable NuStats to provide data from these interviews on an interim schedule. 

A.4.3 Focus Group Field Work 

Because the focus groups are expected to be longer than typical (approximately 4 hours) and emotional in 
content, NuStats will employ a team of experienced, in-house focus group moderators to conduct the 
focus groups with occupants and first responders. These moderators include Dr. Carlos Arce, Dr. 
Johanna Zmud, and Kim Hilsenbeck. Ms. Hilsenbeck will also manage this task, ensuring that all groups 
are organized and scheduled as required. 

DataSource will handle the recruiting, scheduling, and confirmation for all focus groups (for the same 
reasons noted for the face-to-face interviews). Focus groups will be held at MBC Research Center on 

36 NIST NCSTAR 1-7B, WTC Investigation 



Work Plan 



Madison Avenue. A confirmation letter will be mailed to each respondent that explains the purpose of the 
focus group and the protocols in place for identify protection, describes how the session will be 
conducted, and provides directions to the focus group facility. DataSource interviewers will also make all 
the reminder calls. 

Each moderator will employ a moderator protocol for the response population. Discussion guidelines 
provide a general framework for questioning and discussion during a focus group. NuStats will develop 
this guideline in conjunction with NIST and the outside experts. While the focus group moderators 
follow guides as close as possible, each is trained to understand that additional, relevant topics may 
surface during the course of a focus group session. Flexibility is critical to gaining the most valuable 
insight from respondents. If participants diverge into topics that would provide useful data for the study, 
our moderators are experienced in allowing the divergence without going too far off course. 

NuStats anticipates that focus groups will last approximately four hours, with hourly breaks provided. 
The moderators will provide an introduction at the outset of each session, explaining the voluntary nature 
of respondents' participation. This introduction will also provide the basic ground rules for discussion as 
well as disclose any court transcription or viewers. The moderator will also relate the purpose of the 
study and how the focus group results will be used. NuStats expects that there may be questions about 
privacy issues, and we will use this time to reassure all participants about the high security protocols in 
place to protect the identity of and information provided by respondents. 

A.4.4 Status Reporting and Data Deliveries 

Monthly status reports will be delivered to NIST to monitor progress in accordance with the statement of 
work. These reports will advise NIST of work completed during the performance period, and work 
forecasted for the next performance period. NuStats will also provide data deliveries with these status 
reports. Data deliveries (for CATI and face-to-face interviews and transcripts of focus groups) will be 
delivered at the end of September, October, and November. 

NuStats expects to have a meeting with NIST and the outside experts after the delivery of the September 
progress report and data to review the data and any issues that have arisen with the fieldwork. 

A.4.5 Completion of Field Work 

Fieldwork will be deemed complete with NuStats has accomplished all of the sampling requirements set 
forth in this finalized work plan. We expect the vast majority of fieldwork to be completed by the end of 
November. There may be some leakage of field work into early December but expect this to be minimal. 

At the completion of fieldwork, NIST will have received the majority of all required data through the 
interim data deliveries. Three separate databases will be present. Each data set will be cleaned prior to 
delivery. If NIST finds any data cleaning issues, these will be dealt with promptly. To facilitate data 
quality assurance, NuStats will prepare a data quality assurance plan for NIST review prior to the start of 
fieldwork. 

At the completion of all fieldwork, NuStats will prepare a summary analysis for NIST review. The 
summary analysis will contain three parts. The first part is a compilation of on-going (monthly) statistical 
summaries of survey progress, consistency checks, plausibility checks, and other indicators of data quality 
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for the CATI and CAPI data. Second, a summary of the fielding for the CATI and CAPI survey response 
will be summarized, including number of contacts, participation rates, completed responses in each cell of 
the sampling stratification, completed response rates by question. Third a descriptive accounting of the 
CATI and CAPI data will be provided, including frequency distributions and calculations of the mean and 
variance of the sample for all data elements. 

A.4.6 Debrief 

At the conclusion of data collection, NuStats, NIST and the outside experts will meet for a project debrief. 
This meeting will allow for discussion of survey results. From NuStats, participants will include, among 
others, Dr. Johanna Zmud, Project Director, and each of the Task Managers Heather Contrino, Delia 
Santos, and Kim Hilsenbeck. Dr. Carlos Arce and Robert Santos will also be likely attendees from 
NuStats. Prior to coming to this debrief meeting, each of the Task Managers will have debriefed 
surveyors within their span of control. They will compile and bring to the meeting the insights and other 
feedback garnered directly from those involved in the actual interviewing. 

A.4.7 Final Field Work Report 

NuStats will provide two data sets - one for the CATI data and one for the CAPI data. Focus group data 
will be comprised of transcriptions of the focus group sessions. 

In addition, NuStats will prepare a comprehensive draft final report summarizing our work under the 
contract. This final report will fully document the study and sample design including instrument, sample, 
or methodological changes, result of the cognitive interviews and pilot, documentation of data collection 
period and protocols, data processing procedures, and any conclusions. Response rates and call 
dispositions will also be included. In addition, the report will include all materials used in the study 
including the CATI script, CAPI script, focus group protocols, and data file structures. 

NIST will have 30 days to comment on the draft final report. NuStats will incorporate these comments 
any format requirements into the final report and deliver four copies within 15 days of receipt of NIST 
comments. 

A.5 TASK 3: DATABASE COMPLETION AND DELIVERY 

NuStats will deliver the final databases as noted in prior sections of this final work plan. NuStats will not 
conduct any data analysis as part of this contract. Data will be delivered in a format that will be most 
usable to NIST. All analysis will be limited to statistical calculation of the data, and any content analysis 
required to categorize open-ended responses. 

A.5.1 Technical Assistance by Database Expert 

NuStats' database experts will assist NIST after database delivery to facilitate NIST developing an 
understanding of the structure, architecture, limitations, and use of the database (this can include interim 
database delivery if requested by NIST). NuStats will also deliver a Data User's Guide as part of final 
deliverable to NIST. While this Guide will not replace the need for the on-site consultation by our 
database expert, our past experience has shown that it greatly facilitates client use of the database. The 
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Guide will include descriptions of the data file, sample tables, data file codebooks, glossary of terms, 
discussion of estimated sampling errors, and additional background information. 

A.6 SCHEDULE 

The following provides a schedule of project milestones and meetings. 



Task 


Description 


Due Date 


Days from 
Award 




Contract Award 


6/2 





1 


Final Work Plan 


6/16 


13 




CATI and CAPI Instrument Drafts 


7/1 


35 




Meeting with NIST / Experts to Review Drafts 


7/8 


42 




Revised CATI and CAPI Instruments and Focus Guide Protocols 


7/22 


56 




Respondent Materials 


7/22 


56 




IRB Approval 


8/5 


70 




NIST IRB Approval 


8/19 


84 




CATI Pilot Training 


8/21 


86 




CATI Pilot 


8/25 


90 




CAPI Pilot 


8/26-28 


93 




Pre-Field Work Meeting 


8/28-29 


94 




Finalize Instruments and Procedures 


9/15 


110 




Training CAPI 


9/17-18 


113 




OMB Approval 


9/22 


117 




Training CATI 


9/23-24 


119 


2 


Begin CAPI Field Work 


9/23 


119 




Begin CATI Field Work 


10/1 


127 




First Data Delivery and Status Report 


10/3 


129 




First Focus Group Sessions 


10/6-9 


135 




Data Review Meeting 


10/10 


136 




Second Focus Group Sessions 


10/27-30 


156 




Second Data Delivery and Status Report 


11/3 


160 




Third Focus Group Sessions 


11/3-6 


163 




Fourth (Last) Focus Group Sessions 


11/17-20 


177 




End CATI Field Work 


11/27 


184 




Third Data Delivery and Status Report 


12/3 


190 




End CAPI Field Work 


12/12 


199 




Debrief on Field Work and Delivery of Summary Reports 


12/19 


204 




Final Field Work Report 


12/19 


204 


3 


Final Data Delivery 


12/19 


206 




Data Users Guide 


12/19 


206 
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Data Collection Methodology for World Trade Center 

Evacuation and Emergency Response: Telephone Interviews, 

Face-to-Face Interviews, Focus Groups and Population 

Sampling 



1. Background 

The goal of the National Institute of Standards and Technology's World Trade Center Investigation is to 
investigate the building construction, the materials used, and the technical conditions that contributed to 
the outcome of the World Trade Center (WTC) disaster. The results of the Investigation will serve as the 
basis for improvements in the way buildings are designed, constructed, maintained, and used; improved 
tools, guidance for industry and safety officials; revisions to codes, standards, and practices; and 
improved public safety. The primary objectives of the NIST-led technical investigation of the WTC 
disaster are to: 

1. Determine why and how WTC 1 and 2 collapsed following the initial impacts of the aircraft and 
why and how WTC 7 collapsed; 

2. Determine why the injuries and fatalities were so high or low depending on location, including all 
technical aspects of fire protection, occupant behavior, evacuation, and emergency response; 

3. Determine what procedures and practices were used in the design, construction, operation, and 
maintenance of WTC 1, 2, and 7; and 

4. Identify, as specifically as possible, areas in building and fire codes, standards, and practices that 
are still in use and warrant revision. 

The NIST Investigation Plan can be found at http://wtc.nist.gov, including a description of Projects 7 and 
8. Under Project 7, "Occupant Behavior, Egress, and Emergency Communications," first-hand accounts 
of the events of September 11, 2001 from inside WTC 1, 2, and 7 will be collected. This data collection 
effort will evaluate the role of occupant behavior and evacuation technologies and practices for tall 
buildings, including decision-making and situation awareness, time-constrained evacuation strategies, 
communications, role of floor wardens and fire safety directors, and issues concerning people with 
disabilities. Additionally, NIST will seek specific observations of fire and smoke conditions and/or 
structural damage from within the building. Families of the victims, who communicated with loved ones 
inside the Towers before collapse, will be interviewed to determine the nature of the environment above 
the floors of impact. 

The nature of the communications between and among different groups within the World Trade Center 
has been identified as being a potentially significant factor in determining the outcome of the evacuation 
and emergency response. The project will investigate the content and timing of communications among 
the occupants and authorities within the buildings, as well as people outside the buildings. The figure 
below, a hypothetical demonstration of the extraordinary flow of information on the morning of 
September 1 1 th , reinforces the need to understand the role of information transfer in explaining occupant 
and responder actions. In addition to the intergroup communications, communications within each group, 
particularly the building occupants, are potentially important to understanding the events of 
September 11 th . 
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The objectives of Project 8, "Fire Service Technology and Guidelines," are to build upon work already 
done by the Fire Department of New York (FDNY) and McKinsey & Company by: (1) fully documenting 
what happened during the response by the fire services to the attacks on the World Trade Center, up to the 
time of collapse of WTC 7; (2) identifying issues that need to be addressed in changes to practice, 
standards and codes; (3) identifying alternative practices and/or technologies that may address these 
issues; and (4) identifying R&D needs that advance the safety of the fire service in responding to massive 
fires in tall buildings. Thus, a subset of the emergency responders who were present at the World Trade 
Center complex will be asked to voluntarily participate in the face-to-face interview or focus group 
phases of this project. Only first responders who participated in fire suppression, operational, or search 
and rescue activities prior to the building collapse will be considered for inclusion in the population of 
face-to-face interviewees. 

The data collection will be conducted by a yet-to-be-selected contractor and is planned to begin as soon as 
the necessary pre-work is complete. This includes preparation of the telephone interview schedule, face- 
to-face interview protocol, focus group protocol, training of contractor staff, and approval by NIST and 
the appropriate Institutional Review Board (IRB) to assure compliance with federal requirements for the 
protection of human subjects. NIST will use established procedures to review all survey and interview 
questions, data collection methods, and safeguards for maintaining privacy and confidentiality of all 
instruments before proceeding with these critical data collection efforts. 

Note that this paper identifies specific populations and the size of samples to be included in the data 
collection effort. The exact numbers and populations may be modified to better suit the Investigation as 
additional details of the methodology are finalized by NIST and the yet-to-be-chosen contractor. 
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2. Overview of Methodological Approach 

A multidisciplinary, triangulated approach, including telephone interviews, face-to-face interviews, as 
well as focus group interviews has been selected. The multi-methodological approach was selected for 
several reasons. First, multiple methodologies increase confidence in the conclusions and findings when 
more than one methodology arrives at the same conclusions. Second, the varied objectives of the 
Investigation mandate complementary approaches to accomplish all the goals. Finally, concerns 
associated with the time latency since September 11, 2001 suggest the use of different approaches and 
techniques in order to increase memory recall and accuracy. A discussion of each methodology and 
statistical sampling will follow. 

NIST intends to solicit experienced contractors to perform the telephone interviews, face-to-face 
interviews, and focus groups. The contractor will meet or exceed all Federal requirements regarding the 
Common Rule for the Protection of Human Subjects, 1 including Institutional Review Board (IRB) and 
NIST approvals. The objective is to perform up to 600 face-to-face interviews of occupants from areas of 
interest, approximately 150 face-to-face interviews of first responders using selected groups, 
approximately 800 telephone interviews covering selected floors of WTC 1 and WTC 2. Additionally, 
NIST will contract for the conduct of up to 10 focus group sessions with first responders, including the 
Fire Department of New York (FDNY), the New York City Police Department (NYPD), the Port 
Authority Police Department (PAPD), or other groups identified as having operational or command 
authority at the World Trade Center on September 11, 2001. Finally, up to five focus group sessions will 
be conducted with selected building occupants and management. 

2.1 Telephone Interview Format 

The first data collection instrument is the telephone interview. The collection mechanism will be a 
computer-assisted telephone interview. The primary goal of the telephone interview is to provide 
qualitative and quantitative occupant behavior and egress data which can be generalized. A secondary 
goal will be to provide unique, investigative observations, particular to the events at the World Trade 
Center on September 1 1 th . The telephone interview schedule will be closely linked to the evacuation 
experience of the occupants. 

The questions will flow in a logical order in relation to the chronology of the events, as suggested in the 
literature. 2 ' 3 Significant topic areas proposed for the telephone interview include, but are not limited to: 
occupant demographics and inherent traits, chronology of occupant activity, observations and perceptions 
during evacuation, and environmental, social, psychological, physiological, information, frequency, and 
source attributes. As the precise content of the telephone interviews, face-to-face interviews, and focus 
groups has not yet been established, these factors are subject to review and change. 

NIST will follow standard telephone interview construction techniques. 4 These techniques suggest that 
the project team identify the scope and objectives of the telephone interview schedule. The question type 
and format which best accomplishes the scope and objectives will then be selected. The first draft of the 
telephone interview schedule will then be reviewed and revised. Cognitive and pilot testing of the 
telephone interview with the informants will then occur. After further revision, the procedures to 
administer the study will be specified. 

The persons who were in WTC 1 and 2 immediately prior to the first aircraft impact on September 11, 
2001, will constitute the population to be sampled in this study segment, hereafter, to be referred to as the 
"selected floors study." The sampling plan of the September 11, 2001 occupants is a multi-stage 
statistically representative sample with two stages. The first stage will stratify floors by area, population, 
and number of tenants. The second stage will select occupants from the floors selected in the first stage. 

Stratifying the population . Stage one is an area sample of floors. WTC 1 and 2 will be segmented into 
three zones each according locations of the sky lobbies. These zones will approximately represent the top 
(78 th - 1 10 th floors), middle (44 th to 78 th floors), and lower (ground to 43 rd floor) thirds of WTC 1 and 2. 
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This will result in a total of six building zones from WTC 1 and 2. Each zone will then be further 
stratified by tenant number and occupant density. Tenant number will include two levels: single tenant 
floors and multi-tenant floors. Occupant density will include three levels: low, medium, and high density. 
Definitions of low, medium, and high density will be formed after the total building population is 
identified. Thus, each zone will contain six floors, if possible. The second stage is a random sample 
without replacement of occupants from the floors selected in the first stage. 

Enumerating the population . A population list of all the people in each of these 6 building strata 
immediately prior to first impact on September 11, 2001 will then be enumerated. It is estimated that a 
total of 10,000 - 14,000 people were inside WTC 1 and 2 at the time of the first impact, 5 and there are 
unsubstantiated accounts of between 4000 and 5000 persons in WTC 7 on the morning of September 1 1 th . 
For the purposes of sampling and estimation of the number of telephone interviews, face-to-face 
interviews, and focus groups, this project will assume an initial population of 18,500 occupants. NIST 
will provide to the contractor an enumeration of persons on the selected floors. 

Selecting the sample . The sampling plan for the selected floors study will be constructed such that a total 
of 800 telephone interviews are obtained from people included in the study. It is assumed that there will 
be an approximate 30 percent participation rate among those asked to participate. A total of 800 telephone 
interviews is selected since this is twice the estimated number required to obtain a 0.05 level of statistical 
significance. 6 The contractor will make every reasonable effort to increase participation above 30 
percent. 

Data collection . This segment of the study will use a computer assisted telephone interview to obtain data 
from those who choose to participate in the study. 

2.2 Face-to-face interview Format 

The objective of the face-to-face interview segment is to gather first-hand accounts and observations of 
the activities and events inside the buildings on the morning of September 1 1 th . This approach will 
identify unknown information, evaluate technical hypotheses, and explore conscious and subconscious 
motivations for occupant and responder behaviors, while allowing for comparisons to the telephone 
interview data. There will be no verbatim record of the face-to-face interviews, other than random 
selections for quality control purposes. It is estimated that the average face-to-face interview will last 
approximately two hours, with some lasting significantly longer. 

The proposed methodology for the face-to-face interviews is a synthesis of the Behavioral Sequence 
Interview Technique (BSIT) originally developed by Keating and Loftus, 7 and the Cognitive Interviewing 
Method (CIM), originally developed by Fisher 8 and Geiselman. 9 These two interviewing methodologies 
were developed with the purpose of assisting persons in retrieving more comprehensive and accurate 
memories of incidents, and sharing important attributes. Both approaches begin by allowing the informant 
to retell an unimpeded account without interruption from the interviewer, and both initially employ a 
chronological retelling of information. However, BSIT was designed to yield a database of qualitative 
information that could be subjected to systematic analysis and consolidation, while CIM was designed to 
facilitate investigative interviews. Since the Investigation is pursuing both goals (i.e., creation of a 
database of evacuation-related behaviors and an investigatory attempt to capture information relevant to 
outcomes), the proposed methodology combines these two approaches. 

Cognitive interviewing has been the subject of many empirical investigations. Fisher, et al. 10 summarized 
these findings, demonstrating that the methodology significantly increases the amount of information 
recalled without affecting rate of errors. Interviewing a large number of informants will allow 
corroboration of information, thereby compensating for the likely increase in the absolute number of 
errors. Accordingly, it is likely that this approach will be productive in achieving a holistic view of the 
building evacuations. 
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The face-to-face interview methodology, hereinafter referred to as the "areas of interest study," will 
involve face-to-face interviews of occupants and first responders who may have observed (knowingly or 
unknowingly) events important to completion of the objectives of the Investigation. 

Enumerating the population. The population will include the entire occupant, management, and first 
responder population of World Trade Center WTC 1,2, and 7. 

Selecting the sample. The areas of interest sample will identify individuals using the snowball quota 
sample approach whose constituency may resemble individuals selected for the "Specialized Groups 
Study" sampling methodology (see below). A snowball quota sample approach asks individuals for the 
names of other people who may meet the selection criteria for the study. These people are subsequently 
contacted and asked the same question. The process continues until the quota has been reached. The goal 
is to perform approximately 600 face-to-face interviews with occupants, 30 face-to-face interviews with 
family members who communicated with victims inside the building during the event, and 150 first 
responders. The 150 first responders will be divided among the Fire Department of New York 
(firefighters, company officers, and operational command officers), Port Authority Police Department, 
New York Police Department, and other responsible parties. Additional individuals may be randomly 
selected from strata previously defined in the whole buildings study in order to compare the face-to-face 
interview results with the results of the telephone interviews. 

Data Collection. The face-to-face interviews will follow a four step technique, including unimpeded, 
open-ended narrative, a structured narrative, technical probes, and closed-ended questions. Each step is 
described more fully below. 

Step 1: Unimpeded open-ended narrative account. Both BSIT and CIM begin the process by asking 
the participant to chronologically recount his or her "story." The proposed starting point is when it 
became apparent that something unusual had occurred on the morning of September 11, 2001. The 
proposed ending point is when the participant feels that he or she reached a location where they felt safe 
(or, alternatively, when he or she successfully reached the exterior of the building). Researchers and 
practitioners involved with cognitive interviewing believe that starting the face-to-face interviews in this 
manner both improves recall and helps build rapport between the participant and the interviewer. Fisher et 
al. 10 also noted that asking questions may interfere with recall because a participant must divide his or her 
mental resources between recall and listening to the interviewer's questions. 

During the open-ended narrative account, the interviewer can record notable information that can be used 
for the probing phase conducted later. For example, the participant might briefly mention an odd odor to 
which the interviewer will want to return to determine whether the smell might have been that of jet fuel, 
smoke, or of some other origin as yet unknown. 

Step 2: Structured narrative account. After participants complete their stories, interviewers will prompt 
them to go through the story again, but this time they will work cooperatively with the interviewer to 
record entries into a table. This approach is employed by BSIT for three primary reasons: (1) to yield a 
structured account that can be entered into a database without further processing; (2) to avoid the biasing 
effects of having interviewers ask specific questions; and, (3) to enhance the effort at recall put forward 
by participants by encouraging their active collaborative participation, an advantage to open-ended 
formats as noted by Fisher, et al. 10 

Each row of the table will represent a single action in a sentential format, meaning that each action is 
expressed as a grammatical sentence. The approach is used based on the hypothesis that people encode 
episodic memories in a manner consistent with this format, thus facilitating both recall and data entry. 
Each column of the table represents three essential components of actions: a cue, an action, and the reason 
for taking that action. Cues can be either external (e.g., signs of a fire, someone saying something) or 
internal (e.g., remembering about another means of escape.) Actions are expressed using specific action 
verbs (i.e., "ran" instead of "went") and may include artifacts (e.g., a fire extinguisher) used by the 
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informant. Reasons are the intentional, goal-directed base for the action. The interviewer will encourage 
the participant to use their own words to the greatest extent possible. 



A hypothetical example of actions recorded in this manner is: 



Cue 


Action 


Reason 


1 heard but couldn't see 

someone yell "I've found a 
clear path" 


So 1 stumbled in the dark towards 
where 1 thought the voice 
came 


So that 1 could find a way to 
escape 


My path was blocked by debris 


So 1 called out to whoever yelled, 
"I'm near the reception area. 
Where are you?" 


To try to get a better idea 
about where the person 
was 



Table 1: Example Tabular Face-to-face interview Data Entry 

Experimental findings in psychological research on memory 11 suggest that when people perform actions, 
their abilities to verbally recall those actions are significantly improved. Script theory 12 suggests that 
people naturally organize their knowledge of actions using narrative sequences of actions structured 
around their pursuit of goals. However, gaps in the narrative are anticipated, especially given the long 
period of time that will have elapsed between the event and the interview. Interviewers will assist the 
participants to fill in these gaps by asking them to recall events in reverse order, an approach used in 
CIM. Interviewers will, however, encourage participants to report only those memories about which they 
are confident really occurred to them. 

Step 3: Probing for specific information. After completing the structured narrative account, 
interviewers will ask specific open-ended questions (probes) intended to provide specific information of 
particular value to the investigation. While some of this information is likely to be part of the structured 
narrative account, participants may be able to recall other valuable information as well. 

Interviewers may use "context reinstatement" from CIM to improve recall of important information, 
because laboratory experiments have demonstrated that contextual cues enhance recall of related 
information. Fisher et al. explain that context reinstatement may enhance recall because people use 
multisensory coding of events. Using this mnemonic method, interviewers will ask participants to 
"mentally recreate the external environment, and their affective, physiological, cognitive, and emotional 
states that existed at the time of original event." 8 

Depending on the population, probes may be used to try to elicit information including, but not limited to: 

• Location of the informant at the time of certain marker events (e.g., location in WTC 1 when 
WTC 2 collapsed) 

• Fire conditions (e.g., fire and smoke); 

• Other cues of interest (e.g., the smell of jet fuel); 

• Presence and activities of persons with disabilities; 

• Use of elevators by self or others; and, 

• Knowledge of any obstacles to their progress while using the stairs. 

Because information about many of these areas of concern requires precise responses, questions for open- 
ended probes will be developed collaboratively between the contractor and NIST. Responses to probes 
may be recorded using standardized formats where feasible. For example, all participants who observed 
smoke may be asked to estimate the smoke density using an encodable scale, such as visibility distance. 

Step 4: Close-ended telephone interview items. Participants will be asked to complete at least some 
items from the telephone interview. These items are likely to be placed at the end of the face-to-face 
interview session to avoid biasing open-ended responses. The purpose of administering telephone 
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interview items is to determine whether there are systematic differences among the representative samples 
used for telephone interviews, face-to-face interviews, and focus groups. 

Quality Control for Face-to-face interviews. To ensure that interviewers are complying with the face-to- 
face interview techniques and administrative requirements, this project is proposing that interviews be 
videotaped, with the following provisions: the participant signs a form accurately describing the reasons 
for and retention of videotapes and granting permission for the face-to-face interview to be videotaped; 
the contractor will destroy the video record at the conclusion of the investigation; review for quality 
control will be the only purpose for videotaping the face-to-face interview; no transcriptions will be made. 
NIST and the contractor will periodically review videotapes to ensure that interviewers precisely follow 
the protocol and conform to administrative requirements. 

2.3 Focus Groups 

Williams 13 reports that in a group setting, people provide cues that evoke memories in others, and that 
social pressures mediate against reporting misrepresentations of what they recall. Thus, the goal of the 
focus group interviews is to elicit accurate group representations of specific events or themes. Two 
distinct populations will voluntarily participate in the focus groups: occupants and first responders. The 
first set of focus group interviews will be the occupant sample. Distinct categories of people will be 
selected for inclusion in this study, hereafter referred to as the "specialized groups study." The objective 
of this study will be to capture the experience of people in unique places in WTC 1, 2, and 7. These 
groups will be defined by the NIST Investigation team. Every effort will be made to include no less than 5 
people in each of these categories in this study, with 1 people constituting the preferred focus group size. 
NIST anticipates conducting approximately 5 occupant focus groups. 

First responders will constitute a second set of focus group interviews. The set of first responders will 
include FDNY, NYPD, PAPD, and other groups identified as having operational or command authority at 
the World Trade Center on September 1 1 th . The focus group size will be determined as an operating unit 
size, if applicable. An operating unit may be a Fire Department company, for example. This project 
proposes 10 focus groups, each containing 5 people. 

Sample selection . The people selected for inclusion in this study will be selected using non-probability 
sampling procedures. The contractor will use a snowball quota sample. 14 ' 15 Respondents contacted or 
face-to-face interviewed for other reasons will be asked to provide the names and contact information for 
people they know in each of the categories in the specialized groups study. Names will be collected by the 
contractor until at least 5 people in each category have agreed to participate in an occupant focus group, 
with a preference for 10 people. The same process will occur for selection of the first responder samples, 
with a preference for inclusion of entire operating units (about 5 people per unit). 

Data collection . Focus groups will be conducted with the members of each group selected for inclusion in 
each of the specialized categories included in this study segment. The data collected in this study will 
produce qualitative and detailed narrative accounts of the experiences of each category of people. The 
focus group discussion will be moderated by a trained and experienced contractor. 

3. Database 

The contractor will provide to NIST at the conclusion of the project a database of encoded survey results. 
Each telephone interview, face-to-face interview and focus group will result in an encoded table of results 
which can be analyzed using standard data analysis techniques, such as averages, multivariate regression, 
and statistical significance. The specific identity of the encoding variables will be generated jointly by 
the contractor and NIST and is subsequent to the actual content of the survey instruments, which will also 
be developed by the contractor, subject to input and approval from NIST. The number of encoding 
variables is anticipated to be less than 75. This database will need to be consistent with an analysis of 
third-party and media accounts which NIST will generate and code independently of any contractors. 
Analysis of all data and any conclusions derived therein will be the sole responsibility of NIST. 
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However, it is anticipated that a database expert from the survey contractor will assist NIST after database 
delivery in developing an understanding of the structure, architecture, limitations, and use of the database. 

4. Latency and Accuracy of Recall 

The accuracy of participants' memories of events is a consideration, especially given the period of time 
that will have elapsed between the September 1 1 attacks and the data gathering activities. Empirical 
investigations reveal that greater amounts of information are recalled using CI methods without 
increasing the rate of errors. For example, as compared to traditional epidemiological interviews, Fisher et 
al. were able, with CI methods, to elicit many more responses and more precise responses from people 
asked to recall daily physical activities from 35 years earlier. 

NIST will address latency in two ways. First, multiple participants who would have experienced similar 
situations will be used to corroborate as much of the information as possible. Thus, information that 
cannot be reconciled with other evidence may be discounted. Second, the proposed investigative 
approaches are expected to increase the accuracy of the data collected. In a review of research, Pezdek 
and Taylor 16 concluded that people retain fairly accurate memories of directly experienced events. They 
hypothesized that participation in events leads to coherent well-structured narrative memories. Because 
NIST will only be asking about directly experienced events, and will be asking participants to recall 
events in a manner compatible with their naturally occurring internal representations, the accuracy of 
recall should be acceptable. 

5. Protection of Human Subjects 

This data gathering effort will ensure that all precautions required by the Common Rule for the Protection 
of Human Subjects are met or exceeded by the contractor. Participation in any part of this project by any 
person will be strictly voluntary. Interviewers will be trained to establish a rapport with participants 
based on a compassionate interest in their story and will ensure participants that information provided will 
be of value in preventing casualties in future building emergencies. During the briefing, interviewers will 
provide information to participants about where and how to receive counseling without charge, and that 
participants may stop the interview at any time without explanation. Interviewers will also be trained to 
recognize signs of post-traumatic stress. Similar services will be offered to participants of focus groups 
and to people taking the telephone interview. Finally, the contractor will take the necessary precautions 
to ensure the safety of contract employees administering, collecting, or otherwise involved in this data 
collection effort. 

6. Additional Data Collection 

The scenario may arise where an individual critical to developing an understanding of the events of 
September 11, 2001, may be unavailable or unidentified during the period of performance of the 
Contractor. Thus, NIST will obtain NIST IRB approval to conduct a limited number of face-to-face 
interviews with individuals deemed by NIST likely to contribute significantly to the outcome of the 
Investigation. The scope, objectives, and procedures used in the additional data collection will be similar 
to the scope, objectives, and procedures used by the contractor. Telephone interviews and focus groups 
will not be conducted in this additional data collection effort. It is anticipated that the number of face-to- 
face interviews conducted by NIST will be less than 10% of the number of face-to-face interviews 
conducted by the contractor. The contractor will incur no duties or obligations related to the additional 
data collection. 

7. Conclusions and Summary 

Table 1 summarizes the NIST Investigation survey method. NIST proposes a triangulated, 
multidisciplinary survey methodology to analyze and document the events of September 11, 2001, at the 
WTC 1, 2, and 7. The three strategies include telephone interviews, face-to-face interviews, and focus 
groups. The triangulated approach was selected in order to increase confidence in the conclusions, 
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complete dual objectives of generalization and investigation, and increase memory recall and accuracy. 
The methodology and enumeration are summarized below. 

The data collection will be conducted by a yet-to-be-selected contractor and is planned to begin as soon as 
the necessary pre-work is complete. This includes preparation of the telephone interview schedule, face- 
to-face interview protocol, focus group protocol, training of contractor staff, and approval by NIST and 
the appropriate Institutional Review Board (IRB) to assure compliance with federal requirements for the 
protection of human subjects. NIST will use established procedures to review all survey and face-to-face 
interview questions, data collection methods, and safeguards for maintaining privacy and confidentiality 
of all instruments before proceeding with these critical data collection efforts. As additional details of the 
survey methodology are finalized, populations to be included in this project may be modified. 

The telephone interview approach is described as the selected floors study. WTC 1 and 2 will be divided 
into three zones, low, medium, and high. The zones will be further stratified into combinations of single- 
and multi-tenant floors, as well as floors with low, medium, and high occupant densities at the time of the 
first aircraft impact. This is anticipated to result in a 30 percent response rate, yielding approximately 800 
respondents. Eight hundred respondents represent a safety factor of two for the population necessary to 
achieve 0.05 level of statistical significance. 

The areas of interest study will be conducted with face-to-face interviews of up to 600 people. The 
potential respondents will include: 

up to 200 people near floors of impact, 

up to 150 floor wardens, fire safety directors and persons with responsibility, 

up to 100 people in elevators or lobbies, 

up to 100 people from WTC 7, 

up to 30 family members of victims who called out of the towers, and 

up to 20 people with disabilities. 

The Behavioral Sequence Interview Technique and Cognitive Interview Method will be combined in the 
face-to-face interview sessions. This approach will maximize the investigative return in order to identify 
unknown information, evaluate technical hypotheses, and explore conscious and subconscious 
motivations for occupant and responder behaviors, while allowing for comparisons to the telephone 
interview data. NIST will also conduct face-to-face interviews with members of the Fire Department of 
New York, Port Authority Police Department, New York Police Department, and others having 
operational responsibilities. This approach will face-to-face interview approximately 150 people, with the 
population being stratified among firefighters, company officers, and operational command officers. 

The third approach will employ focus groups. NIST anticipates creating five focus groups of building 
occupants with approximately 10 people per group. The population will be generated using the snowball 
quota sample approach. Additionally, NIST will create approximately 1 focus groups with first 
responders, with each focus group containing approximately five individuals. The population will be 
generated using the snowball quota sample approach. 



NIST NCSTAR 1-7B, WTC Investigation 49 



Appendix B 





Intended 

Number of 

Respondents 


Intended 

Response 

Rate 






Method 


Sampling , 

„ Population 

Strategy 




Telephone 
Questionnaire 


Selected Floors 
(Occupants) 


800 


30% 


Statistically 

Representative 

Area Sampled 

Floors 


WTC 1 and 2 


CO 

'> 
CD 

s 

<D 

o 

<S 

o 

<D 

o 

CS 

Ph 


Areas of Interest 
(Occupants) 


570 


N/A 


Snowball Quota 
and Randomly 

Selected 


WTC 1,2, 

and 7 


Areas of Interest 
(Families) 


30 


N/A 


Snowball Quota 


WTC 1 and 2 


Areas of Interest 
(First Responders) 


150 


N/A 


Snowball Quota 


FDNY, 

NYPD, 

PAPD, others 


CO 

o 

in 

o 

CO 

o 
O 


Specialized 

Groups 
(Occupants) 


50 


N/A 


Snowball Quota 


WTC 1,2, 

and 7 


Specialized 
Groups (First 
Responders) 


50 


N/A 


Snowball Quota 


FDNY, 
NYPD, 

PAPD, others 



Table 2: Summary of Methods 
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